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Factors  Influencing  the  Psychometric  Characteristics 
of  an  Adaptive  Testing  Strategy  for  Test  Batteries 


Recent  studies  (e.g.,  Bejar,  Weiss,  &  Gialluca,  1977;  Be jar,  Weiss,  & 
Kingsbury,  1977)  have  demonstrated  the  feasibility  of  applying  unidimensional 
adaptive  testing  strategies  to  measure  classroom  achievement.  Frequently,  how¬ 
ever,  achievement  test  batteries  are  composed  of  items  drawn  from  several  dis¬ 
tinct  content  or  subject  matter  areas.  Under  these  circumstances,  the  unidimen¬ 
sionality  assumption  may  be  untenable  or  inappropriate  when  applied  to  the  en¬ 
tire  set  of  items.  Application  of  a  unidimensional  item  response  theory  (IRT) 
model  to  multidimensional  test  data  usually  results  in  achievement  level  esti¬ 
mates  that  reflect  achievement  on  only  a  small  subset  of  the  test  domain  (i.e., 
that  subset  of  items  having  loadings  on  the  first  factor).  Thus,  for  example, 
although  Bejar,  Weiss,  and  Kingsbury  (1977)  showed  that  a  single  factor  emerged 
from  analysis  of  a  multicontent  achievement  test,  that  factor  accounted  for  only 
23.3%  of  the  variance  of  the  original  variables.  As  a  consequence,  unidimen¬ 
sional  adaptive  testing  using  that  single  factor  would  leave  76.7%  of  the  origi¬ 
nal  variance  of  the  items  unaccounted  for.  Similar  results  were  reported  by 
Reckase  (1978),  who  showed  first  factors  of  achievement  tests  accounting  for 
1.6%  to  81.4%  of  the  variance  of  the  original  items. 

Treating  multicontent  achievement  test  batteries  as  if  they  were  unidimen¬ 
sional  also  results  in  loss  of  diagnostic  information  about  a  student  by  reduc¬ 
ing  data  on  a  student  to  only  one  score.  Frequently,  however,  scores  on  each 
content  area  are  important  information  to  be  used  by  instructors  for  instruc¬ 
tional  decisions  at  both  the  individual  and  class  level. 

Brown  and  Weiss  (1977)  designed  an  adaptive  testing  strategy  for  use  with 
test  batteries  that  would  reduce  testing  time,  yet  provide  scores  on  the  sepa¬ 
rate  subtests  in  the  battery.  Their  testing  strategy  included  provision  for 
adaptively  branching  between  content  area  subtests  as  well  as  adaptive  item  se¬ 
lection  within  a  content  area  subtest.  Brown  and  Weiss  investigated  the  charac¬ 
teristics  of  their  combined  inter-subtest/intra-subtest  adaptive  testing  strate¬ 
gy  using  a  real-data  simulation  of  a  military  achievement  test  battery.  In  this 
approach,  item  response  data  obtained  under  conventional  testing  conditions  were 
reanalyzed  as  if  they  were  administered  by  the  adaptive  testing  strategy.  The 
findings  indicated  average  reductions  in  test  battery  length  of  approximately 
50%  while  maintaining  high  levels  of  psychometric  information.  Unfortunately, 
Brown  and  Weiss's  (1977)  results  confounded  the  relative  contributions  of  intra¬ 
subtest  item  selection  and  inter-subtest  branching. 

In  an  attempt  to  isolate  the  separate  effects  of  the  intra-subtest  and  in¬ 
ter-subtest  components  of  the  Brown  and  Weiss  testing  paradigm,  Gialluca  and 
Weiss  (1979),  also  using  a  real-data  simulation  design,  applied  the  testing 
strategy  to  a  five-subtest  biology  test  battery.  Their  findings  showed  test 
length  reductions  of  20%  to  30%,  again  with  minimal  loss  of  psychometric  infor¬ 
mation.  They  concluded  that  most  of  the  reduction  in  test  length  was  due  to  the 
adaptive  intra-subtest  item  selection  procedure  and  that  the  addition  of  inter- 


subtest  branching  produced  an  additional  reduction  of  1%  to  5%  in  test  length. 

Both  Brown  and  Weiss  (1977)  and  Gialluca  and  Weiss  (1979)  evaluated  the 
performance  of  the  adaptive  testing  strategy  in  similar  ways.  One  of  their 
evaluative  criteria  was  correlations  of  achievement  estimates  from  the  adaptive 
tests  with  those  from  the  original  conventional  tests.  Evaluation  of  these  cor¬ 
relations  is  difficult,  however,  since  they  are  part-whole  correlations  that  are 
artifically  inflated  due  to  the  items  administered  in  common  by  the  two  testing 
strategies.  Thus,  although  the  previous  research  has  demonstrated  essentially 
no  loss  in  psychometric  information  due  to  the  adaptive  testing  strategy,  the 
question  of  the  effect  on  score  validity  has  not  been  investigated.  Previous 
research  also  was  restricted  to  the  application  of  only  one  method  of  ordering 
subtests  for  adaptive  administration:  subtests  in  both  studies  were  ordered  by 
their  multiple  correlations  with  each  other. 

Purpose 

The  present  study  Investigated  the  separate  contributions  of  intra-subtest 
adaptive  item  selection  and  lnter-subtest  adaptive  branching  in  terms  of  (1) 
reduction  in  the  number  of  items  administered,  (2)  psychometric  information 
available  in  the  test  scores,  and  (3)  correlations  between  achievement  estimates 
derived  from  adaptive  and  conventional  test  administration  with  true  achievement 
levels.  In  addition,  different  methods  for  ordering  the  subtests  for  adaptive 
administration  were  studied. 


METHOD 

Procedure 


Test  Items 


Monte  carlo  simulation  was  used  so  that  the  validity  question,  i.e.,  the 
correlation  of  observed  with  true  ability  estimates,  could  be  adequately  inves¬ 
tigated.  Since  a  simulation  study  is  valuable  to  the  extent  that  the  underlying 
model  accurately  reflects  the  characteristics  of  actual  data,  the  achievement 
test  data  used  by  Brown  and  Weiss  (1977)  formed  the  basis  for  this  study. 

The  Brown  and  Weiss  data  consisted  of  the  response  vectors  of  365  Navy  fire 
control  technicians  on  a  232-item  achievement  test  battery.  The  test  battery 
was  composed  of  12  subtests,  each  covering  a  different  subject  matter  area.  The 
items  were  parameterized  independently  for  each  subtest  using  Urry's  (1976,  p. 
99)  Ogivia  computer  program  utilizing  the  three-parameter  normal  ogive  model. 

Table  1  shows  the  means  and  standard  deviations  of  the  IRT  item  parameters 
£,  1>,  and  £  from  the  Brown  and  Weiss  (1977)  Ogivia  parameterization.  (Individ¬ 
ual  item  parameter  estimates,  by  subtest,  are  in  Appendix  Table  A.)  Columns  2, 
3,  and  4  of  Table  1  show  that,  from  the  total  available  item  pool  of  232  items, 
item  parameter  estimates  were  obtained  for  87%  (or  201)  of  these  items.  Subtest 
12  lost  the  greatest  number  of  items  (28%  of  the  original  25),  whereas  Subtests 
1,  2,  6,  and  8  did  not  lose  any  of  their  items. 


Mean  item  difficulty  (b)  ranged  from  .06  for  Subtest  1  to  1.44  for  Subtest 


Table  1 

Means  and  Standard  Deviations  of  Normal  Ogive  Item  Discrimination  (a). 
Difficulty  (b),  and  Pseudo-Guessing  ( c )  Parameters 
for  12  Subtests  (from  Brown  &  Weiss,  1977) 


Subtest 

Number 

Avail¬ 

able 

of  Items 
Parame¬ 
terized 

Percent 

of  Items 
Parame¬ 
terized 

a 

Mean 

SD 

Parameter 
b  c 

Mean  SD  Mean 

SD 

1 

10 

10 

100 

1.90 

.62 

.06 

1.03 

.52 

.11 

2 

10 

10 

100 

2.12 

.86 

.31 

1.29 

.53 

.18 

3 

18 

15 

83 

1.80 

.56 

.54 

1.30 

.55 

.08 

4 

22 

19 

86 

1.60 

.60 

.43 

1.28 

.47 

.08 

5 

18 

17 

94 

1.57 

.65 

.74 

1.32 

.47 

.10 

6 

18 

18 

100 

1.58 

.43 

1.19 

1.45 

.56 

.09 

7 

14 

13 

93 

1.98 

.94 

1.20 

1.26 

.52 

.18 

8 

12 

12 

100 

2.12 

.90 

.84 

1.10 

.43 

.10 

9 

24 

22 

92 

1.49 

.59 

.88 

1.36 

.43 

.10 

10 

29 

23 

79 

1.66 

.57 

1.28 

1.12 

.44 

.14 

11 

32 

24 

75 

1.48 

.61 

.91 

1.39 

.43 

.14 

12 

25 

18 

72 

1.73 

.58 

1.44 

1.34 

.52 

.17 

Total 

232 

201 

87 

1.75 

.54 

.82 

1.38 

.49 

.10 

and  mean 

item  discrimination 

(a)  ranged 

from 

1.48 

for  Subtest  11 

to  2 

.12  for 

Subtests  2  and  8.  Mean  estimates  of  the  pseudo-guessing  parameter  (£)  ranged 
from  .43  for  Subtests  8,  9,  and  11  to  .56  for  Subtest  6. 

Generation  of  Simulated  Examinees 

The  first  step  in  a  simulation  is  generation  of  hypothetical  examinees  with 
trait  levels  matching  known  or  assumed  characteristics  of  a  real  population. 
Using  the  subtest  score  intercorrelation  matrix  from  Brown  and  Weiss  (1977,  p. 
11),  12  trait  levels  were  generated  for  each  examinee  by  the  following  algo¬ 
rithm.  Twelve  principal  components  were  extracted  from  the  subtest  intercorre¬ 
lation  matrix.  Independently  distributed  standard  normal  deviates  were  drawn 
from  a  random  number  generator  and  assigned,  one  to  each  component.  An  examin¬ 
ee's  trait  score  was  considered  to  be  the  sum  of  the  product  of  the  component 
loading  for  that  trait  on  each  of  the  components,  multiplied  by  the  random  devi¬ 
ate  assigned  to  that  component.  In  matrix  notation,  this  can  be  written  as 


where  Z  is  a  1  x  12  matrix  of  standard  normal  deviates  and  W  is  a  12  x  12  matrix 
of  factor  loadings. 

A  1,000  y  12  (examinee-by-subtest)  matrix  of  generated  true  achievement 
levels  was  obtained  for  computing  the  subtest  intercorrelation  matrix  to  be  used 
for  ordering  the  subtests  by  multiple  correlations.  A  random  sample  of  300  ex¬ 
aminees  was  retained  for  the  testing  simulations.  Using  this  examinee-by-sub¬ 
test  matrix  of  generated  true  achievement  levels,  the  multivariate  item  response 
matrix  was  generated. 


Item  Response  Generation 

Response  vectors  for  simulated  examinees  were  generated  based  on  the  item 
parameters  for  items  measuring  several  latent  traits.  Given  0^  (simulee  i's 

true  ability  on  trait  k)  and  the  parameters  for  item  g  on  trait  k,  the  probabil¬ 
ity  of  simulee  i  responding  correctly  was  calculated  as 


P  .  (0,,  )  =  c  ,  +  (1  -  c  )  Y  [Da  ,  (6.,  -  b  .  )] 

gkv  ik  gk  gk'  gk  ik  gk' 

where 

ag^  is  the  discrimination  for  item  g  measuring  trait  k, 

bgfc  is  the  difficulty  of  item  g  on  trait  k, 

cgk  t*,e  Pseudo-8ue88in8  level  of  item  £  on  trait  k, 

D  is  1.7,  a  scaling  constant  for  the  logistic  model, 
and  <f[x]  is  the  logistic  cumulative  distribution  function. 


[2] 


The  probabilities  thus  computed  were  compared  to  a  vector  of  uniformly  dis¬ 
tributed  random  numbers,  ru,  in  the  interval  0  to  1.  If 


or  if 


r  <  P  .  (0..  ),  u  =  1 
u  -  gk  ik  *  gik 


r  >  P  ,  (0.,  ),  u  -  0. 
u  gk  ik  gik 


[3] 


[41 


Conventional  Test 


To  provide  a  basis  for  comparison  of  the  adaptive  testing  strategy,  conven¬ 
tional  administration  of  the  achievement  battery  was  also  simulated.  The  sub¬ 
tests  were  administered  sequentially,  with  all  items  within  a  subtest  given  in 
order  and  all  examinees  taking  all  the  items  in  the  same  order.  Owen's  (1975) 
Bayesian  scoring  algorithm  was  used  to  score  each  of  the  subtests,  with  a  mean 
of  0.0  and  variance  of  1.0  as  the  initial  prior  estimate  for  each  subtest.  That 
is,  no  differential  item  selection  or  differential  subtest  order  occurred  for 
the  conventional  test. 


Adaptive  Testing  Strategy 

As  in  the  Brown  and  Weiss  (1977)  and  Gialluca  and  Weiss  (1979)  studies,  an 
adaptive  testing  strategy  utilizing  both  intra-subtest  item  selection  based  upon 
the  item  information  function  and  a  regression-based  inter-subtest  branching 
rule  were  used.  Three  ways  of  ordering  the  subtests,  for  use  with  the  regres¬ 
sion  approach  to  estimate  differential  subtest  entry,  were  simulated:  (1)  or¬ 
dering  on  the  basis  of  highest  multiple  correlation  (as  in  the  two  previous 
studies),  (2)  ordering  on  the  basis  of  number  of  items  in  the  subtest,  and  (3) 
random  ordering. 

Additionally,  an  adaptive  testing  strategy  employing  only  intra-subtest 
item  selection  was  simulated.  This  was  done  in  order  to  separate  the  effects  of 
variable  termination  in  the  intra-subtest  item  selection  strategy  from  those  of 
the  inter-subtest  branching  strategy.  Instead  of  differential  entry  into  subse¬ 
quent  subtests  based  on  information  from  previous  subtests,  each  subtest  was 
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treated  individually,  as  in  the  conventional  test.  As  with  the  conventional 
test,  Bayesian  scoring  was  used,  with  a  mean  of  0.0  and  variance  of  1.0  as  the 
initial  prior  §  for  each  of  the  subtests.  Hence,  the  only  difference  between 
these  tests  and  the  other  adaptive  tests  was  that  inter-subtest  branching  (and, 
therefore,  a  differential  Bayesian  prior  achievement  estimate  for  each  simulee 
on  each  subtest)  was  not  used. 

Intra-Subtest  Adaptive  Testing 


Item  selection.  Adaptive  intra-subtest  item  selection  used  a  maximum  in¬ 
formation  item  selection  method  in  which  the  item  to  be  administered  to  each 
simulee  at  each  stage  of  the  test  was  that  item  which  provided  the  most  informa¬ 
tion  at  that  simulee's  current  achievement  level  estimate  (see  Brown  &  Weiss, 
1977,  for  a  detailed  explanation  of  this  method).  Hence,  for  each  simulee,  the 
item  information  value  for  each  unadministered  item  within  a  subtest  was  evalu¬ 
ated  *:rom 

[p'(0)l2 

I(0)  =  Pg(0) [l  -  prC8)]  151 

where  P  (6)  is  defined  as  in  Equation  2  and  P'  is  the  first  derivative  of  the 

8  A 

item  characteristic  curve  evaluated  at  0 ,  the  simulee's  current  achievement  level 
estimate.  The  item  that  had  the  highest  information  value  was  selected  for  ad¬ 
ministration;  once  an  item  was  administered  to  a  simulee,  it  was  omitted  from 
further  consideration  for  that  simulee. 

Scoring  and  termination.  As  in  the  conventional  test,  Bayesian  achievement 
level  estimates  were  obtained  for  each  simulee  after  administration  of  each 
item.  Termination  of  testing  within  a  subtest  was  governed  by  two  criteria:  (1) 
testing  terminated  when  all  items  within  a  subtest  had  been  administered  or  (2) 
when  no  item  remaining  in  a  subtest  provided  a  predetermined  amount  of  informa¬ 
tion  at  the  current  estimate  of  0.  Two  minimum  values  of  information  were  used 
in  this  study:  .01  and  .05. 

Inter-Subtest  Adaptive  Branching 


Subtest  ordering.  Three  methods  of  ordering  the  subtests  were  simulated. 

In  the  first  method,  adapted  from  Brown  and  Weiss  (1977),  subtests  were  ordered 
on  the  basis  of  their  highest  multiple  correlation  with  each  other.  Brown  and 
Weiss,  however,  ordered  subtests  based  on  linear  regression  of  number-correct 
scores.  In  this  study,  the  simulee-by-subtest  matrix  of  generated  true  achieve¬ 
ment  levels  (N  =  1,000)  was  intercorrelated  and  the  resulting  correlation  matrix 
was  used  as  the  basis  for  inter-subtest  branching.  The  highest  zero-order  cor¬ 
relation  was  chosen  from  this  intercorrelation  matrix,  one  of  the  two  subtests 
was  arbitrarily  chosen  to  be  administered  first,  and  the  other  was  administered 
second.  Multiple  correlations  were  then  computed  using  the  subtests  previously 
chosen  first  and  second  as  predictors,  with  each  of  the  remaining  subtests,  in 
turn,  being  designated  as  the  criterion.  The  subtest  having  the  highest  corre¬ 
lation  with  the  predictor  subtests  was  chosen  to  be  administered  next.  By  add¬ 
ing  one  subtest  to  the  predictor  set  at  each  succeeding  stage,  all  subtests  were 
thus  o_dered. 
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Ordering  subtests  on  the  basis  of  this  stepwise  regression  procedure  re¬ 
quires  [N(N-l)/2]-N  regression  equations,  whereas  given  an  a  priori  subtest  or¬ 
dering,  only  N-l  regression  equations  are  required  to  estimate  differential  in¬ 
dividual  entry  achievement  estimates.  Although  the  ordering  of  subtests  by 
stepwise  linear  regression  appears  to  be  a  natural  adjunct,  two  other  procedures 
were  implemented  so  that  the  stepwise  procedure  could  be  evaluated  against  al¬ 
ternatives. 

One  alternative,  suggested  by  Brown  and  Weiss  (1977),  was  to  order  subtests 
based  on  the  number  of  items.  It  seems  logical  to  administer  the  longer  sub¬ 
tests  first,  since  at  the  early  stages  of  testing  a  wider  range  of  items  would 
enable  a  more  accurate  assessment  of  achievement  levels.  As  the  differential 
entry  achievement  estimates  became  more  accurate,  fewer  items  would  be  required 
to  assess  the  examinee's  achievement  levels  on  subsequent  tests. 

Subtest  order  may  have  an  effect  on  the  psychometric  characteristics  of  the 
adaptive  testing  strategy  or  on  number  of  items  administered.  If  it  does  not, 
then  it  should  be  possible  to  obtain  the  same  results  by  ordering  subtests  ran¬ 
domly.  To  test  this  hypothesis,  subtests  were  ordered  randomly  as  a  third  way 
of  ordering  them  for  adaptive  administration. 

Differential  subtest  entry.  After  administration  of  the  first  subtest, 
entry  achievement  estimates  for  subsequent  subtests  were  differentially  deter¬ 
mined  for  each  simulee.  For  the  first  subtest,  however,  each  simulee's  entry 
achievement  estimate  was  determined  by  setting  its  prior  achievement  level  esti¬ 
mate  to  be  0  *  0.0  and  selecting  that  item  from  the  initial  subtest  that  provid¬ 
ed  the  most  information  at  0  *  0.0.  Thus,  all  simulees  began  the  initial  sub¬ 
test  with  the  same  item. 

Entry  into  the  item  pool  for  the  next  subtest  was  determined  by  the  bivari¬ 
ate  regression  of  scores  from  that  subtest  on  the  first  subtest  and  the  simu¬ 
lee's  estimated  achievement  level  on  the  first  subtest.  This  yielded  an  esti¬ 
mate  of  the  simulee's  achievement  level  on  the  second  subtest.  This  achievement 
level  estimate  then  became  the  Bayesian  prior  for  intra-subtest  item  selection 
for  the  second  subtest.  The  squared  standard  error  of  estimate  from  the  bivari¬ 
ate  regression  equation  became  the  Bayesian  prior  variance. 

In  general,  an  examinee's  final  achievement  level  estimates  from  all  ti  pre¬ 
viously  administered  subtests  were  used  in  the  appropriate  linear  multiple  re¬ 
gression  equation  for  predicting  the  ii  +  1st  subtest  score.  The  squared  stan¬ 
dard  error  of  estimate  from  each  regression  was  used  as  the  Bayesian  prior  vari¬ 
ance  for  that  subtest. 


Dependent  Variables 


Validity  and  Test  Length 

In  a  simulation  study,  true  or  generated  ability  or  achievement  levels  es¬ 
tablish  the  criterion.  Thus,  correlations  of  achievement  level  estimates  on 
each  subtest  with  their  known  true  values  were  obtained  for  all  testing  condi¬ 
tions.  In  addition,  the  number  of  items  administered  under  each  testing  condi¬ 
tion  was  examined  for  each  subtest. 


Test  Information 
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The  precision  of  measurement  at  different  achievement  levels  is  indexed  by 
its  information  value,  a  quantity  that  is  inversely  proportional  to  the  squared 
length  of  the  confidence  interval  about  an  achievement  level  estimate  (Hambleton 
&  Cook,  1977).  Subtest  information  curves  were  computed  by  evaluating  the  item 
information  function  at  each  simulee's  estimated  achievement  level  and  summing 
over  all  items  administered  in  that  subtest.  Examinees  were  then  grouped  into 
equally  spaced  nonoverlapping  intervals,  and  the  mean  information  over  all  simu- 
lees  within  an  interval  was  plotted  at  the  midpoint  of  that  interval  to  obtain 
the  subtest  information  curves. 

Statistics  descriptive  of  information  curves.  Although  graphs  are  a  simple 
and  convenient  way  to  present  information  curves,  they  are  difficult  to  inter¬ 
pret  directly  when  many  curves  are  involved.  Thus,  for  comparison  purposes,  the 
means  and  coefficients  of  variation  of  the  information  curves  were  computed. 

Mean  or  average  information  is  a  statistic  that  is  not  disproportionately 
weighted  by  the  distributional  characteristics  of  the  examinee  group  tested. 

The  greater  the  mean  information,  the  more  precise  are  the  achievement  level 
estimates,  on  the  average,  over  all  levels  of  achievement.  The  coefficient  of 
variation  (Snedecor  &  Cochran,  1967,  p.  62),  which  is  equal  to  the  standard  de¬ 
viation  of  the  information  curve  divided  by  its  mean,  is  of  interest  because  its 
departure  from  zero  indicates  that  the  goal  of  equiprecise  measurement  is  not 
being  met. 

Relative  information.  Relative  information  may  be  defined  as 

RIA/B  =  V0)/lB(e>  [61 

where  1^(0)  is  the  information  value  of  test  A  and  lg(0)  is  the  information  val¬ 
ue  of  test  B  (Be jar,  1977).  A  useful  interpretation  is  in  terms  of  test  length 
(Lord,  1980):  when  Rl^/g  >  1-00  at  a  given  achievement  level,  test  A  provides 

an  amount  of  information  equal  to  test  B  lengthened  by  a  factor  of  Rl^y^ .  In 

the  present  study,  for  example,  relative  information  was  computed  for  each  adap¬ 
tive  subtest  relative  to  its  conventional  counterpart.  That  is,  the  numerator 
was  the  value  of  information  for  the  adaptive  subtest  and  the  denominator  was 
the  value  of  information  for  that  same  subtest  administered  conventionally. 


RESULTS 


Subtest  Order 


The  intercorrelations  of  the  simulee-by-subtest  matrix  of  generated  true 
achievement  levels  are  shown  in  the  lower  triangle  of  Table  2.  The  residuals 
from  the  target  correlation  matrix  (Brown  &  Weiss,  1977,  p.  11)  are  shown  in  the 
upper  triangle  of  Table  2.  The  largest  absolute  residual  was  .08.  The  average 
absolute  deviation  from  the  target  matrix  was  .03,  so  that,  overall,  the  target 
matrix  appeared  to  be  reproduced  faithfully.  However,  the  reconstructed  corre¬ 
lation  matrix  was  sufficiently  different  to  change  the  rank  ordering  of  the  sub¬ 
tests  based  on  the  highest  multiple  correlations  with  each  other,  from  the  rank 
orderings  obtained  by  Brown  and  Weiss  (1977).  The  highest  bivariate  correlation 
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(.52)  was  observed  between  Subtests  3  and  11,  which  were  designated  to  be  admin¬ 
istered  first  and  second,  respectively. 


Table  2 

Intercorrelations  Among  True  Achievement  Levels  (Lower  Triangle) 
and  Residuals  from  Generated  Correlation  Matrix  (Upper  Triangle) 


Subtest 

Subtest 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

1 

-02 

00 

01 

-04 

-05 

-01 

-04 

-01 

-03 

-01 

-02 

2 

29 

-07 

-05 

00 

-04 

-01 

-01 

00 

01 

-05 

06 

3 

40 

30 

-04 

-01 

-05 

00 

-02 

-04 

-02 

-01 

-01 

4 

37 

35 

42 

-02 

-04 

-03 

-05 

-06 

-06 

-02 

-01 

5 

34 

37 

47 

36 

-05 

00 

-03 

00 

00 

-03 

00 

6 

25 

22 

31 

35 

33 

-01 

-06 

-01 

-04 

-08 

-01 

7 

29 

37 

41 

33 

46 

34 

-03 

02 

-01 

02 

-01 

8 

21 

28 

27 

23 

32 

24 

33 

-02 

-02 

-04 

01 

9 

22 

33 

38 

42 

47 

44 

43 

26 

-05 

-01 

-04 

10 

16 

36 

25 

27 

28 

29 

32 

25 

35 

-04 

01 

11 

41 

28 

52 

37 

38 

22 

39 

24 

34 

23 

06 

12 

25 

33 

21 

13 

29 

15 

26 

27 

22 

32 

32 

Note.  Decimal  points  omitted. 


Multiple  regression  equations  were  obtained  by  using  Subtests  3  and  11  as 
predictor  variables  and  each  of  the  remaining  subtests,  in  turn,  as  criterion 
variables.  This  procedure  continued  until  all  the  subtests  had  been  ordered. 
Table  3  contains  the  multiple  correlations  for  each  subtest  predicted  from  all 
previous  subtests  and  the  subsequent  ordering  of  the  subtests  based  on  the  mul¬ 
tiple  correlations.  The  raw  score  regression  weights,  regression  constants,  and 
squared  standard  errors  of  estimate  are  shown  in  Appendix  Table  B.  Table  4 
shows  the  ordering  sequence  of  each  subtest  based  on  the  following  three  order- 

Table  3 

Multiple  Correlations  Among  Ordered  Subtests 


Criterion  Predictor  Subtest 


Subtest 

3 

11 

5 

7 

9 

4 

1 

6 

2 

11 

52 

5 

47 

50 

7 

41 

46 

54 

9 

38 

41 

51 

55 

4 

42 

45 

48 

49 

53 

1 

40 

46 

48 

48 

49 

57 

6 

31 

32 

38 

41 

49 

51 

51 

2 

30 

33 

41 

45 

46 

48 

49 

49 

10 

25 

27 

32 

36 

41 

42 

42 

42 

47 

12 

21 

32 

37 

38 

38 

38 

40 

40 

44 

8 

27 

29 

36 

39 

40 

40 

40 

41 

42 

Note.  Decimal  points  omitted 
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ing  met ho ding  (1)  highest  multiple  correlation,  (2)  number  of  items,  and  (3) 
random.  Regression  weights  and  related  data  for  the  latter  two  ordering  methods 
are  in  Appendix  Tables  C  and  D. 


Table  4 

Order  of  Administration  of 
12  Subtests  Resulting  from 
Three  Ordering  Methods 


Ordering  Method 

Highest  Number  of 

R 

Items 

Random 

3 

11 

3 

11 

10 

4 

5 

9 

11 

7 

4 

2 

9 

12 

7 

4 

6 

6 

1 

5 

1 

6 

3 

5 

2 

7 

8 

10 

8 

10 

12 

2 

12 

8 

1 

9 

Test  Length 

Summary  statistics  for  the  number  of  items  administered  in  each  of  the  12 
subtests  for  each  subtest  order  and  inter-subtest  branching  condition,  and  two 
levels  of  intra-subtest  termination  are  given  in  Tables  5  and  6. 

Adaptive  intra-subtest  item  selection.  The  data  in  Table  5  summarize  the 
reductions  in  mean  test  length  observed  when  subtests  were  administered  adap¬ 
tively  but  with  no  inter-subtest  branching.  That  is,  each  subtest  was  adminis¬ 
tered  as  an  independent  subtest  using  only  adaptive  item  selection  within  each 
subtest.  Thus,  each  examinee  began  each  subtest  with  a  Bayesian  prior  achieve¬ 
ment  estimate  of  0.0  and  a  prior  variance  of  1.0.  The  length  of  the  total  test 
battery  averaged  95.78  items  under  the  termination  criterion  of  .05  and  115.55 
items  under  the  .01  termination  criterion. 

The  maximum  number  of  items  administered  for  the  total  battery  using  only 
adaptive  intra-subtest  item  selection  with  .05  termination  was  154,  which  was  a 
23%  reduction  in  total  test  length;  for  the  .01  termination  condition,  the  maxi¬ 
mum  number  of  items  administered  was  171,  which  was  a  15%  reduction  in  total 
test  battery  length.  The  shortest  adaptive  battery  yielded  a  78%  reduction  in 
teat  length  for  .05  termination  (44  items)  and  a  72%  reduction  for  .01  termina¬ 
tion  (56  items).  The  average  reduction  in  test  length  over  the  total  test  bat¬ 
tery  (i.e.,  the  average  of  the  subtest  mean  reductions  weighted  by  the  number  of 
items  in  each  subtest)  was  52.3%  for  the  .05  termination  and  42.5%  for  the  .01 
termination. 


Table  5 

Number  of  Items  Administered  in  12  Adaptive  and  Conventional  Subtests 
with  Intra-Subtest  Item  Selection  Only  at  Two  Termination  Levels 


Sub¬ 

test 

Conven¬ 

tional 

Test 

Termination 

Criterion 

of  .05 

Termination 

Criterion 

of  .01 

Mean 

SD 

Range 
Min  Max 

Percent 

Reduc¬ 

tion 

Mean 

SD 

Range 
Min  Max 

Percent 

Reduc¬ 

tion 

1 

10 

6.07 

1.23 

3 

8 

39.3 

7.29 

.87 

5 

8 

27.1 

2 

10 

6.12 

1.00 

3 

8 

38.8 

7.00 

.80 

4 

8 

30.0 

3 

15 

7.61 

1.42 

3 

11 

49.3 

9.12 

1.50 

4 

12 

39.2 

4 

19 

10.93 

2.20 

6 

15 

42.5 

12.99 

1.89 

6 

15 

31.6 

5 

17 

9.12 

2.30 

5 

12 

46.4 

10.98 

2.60 

6 

14 

35.4 

6 

18 

7.55 

1.99 

3 

10 

58.0 

8.78 

2.03 

4 

15 

51.2 

7 

13 

5.19 

1.60 

2 

12 

60.1 

5.63 

1.42 

2 

12 

56.7 

8 

12 

5.88 

2.05 

3 

10 

51.0 

7.16 

2.14 

3 

11 

40.4 

9 

22 

11.73 

2.68 

6 

17 

46.7 

14.09 

2.90 

8 

19 

35.9 

10 

23 

7.54 

4.14 

3 

21 

67.2 

10.78 

4.58 

3 

21 

53.1 

11 

24 

12.09 

2.58 

5 

17 

49.6 

14.52 

1.82 

8 

20 

39.5 

12 

18 

5.95 

1.99 

2 

13 

66.9 

7.21 

2.14 

3 

16 

60.0 

Total 

Battery 

201 

95.78 

44 

154 

52.3 

115.55 

56 

171 

42.5 

♦Computed  by  the  formula  100-((mean  number  of  items  in  adaptive  test/number 
of  items  in  conventional  test)  x  100J 


The  largest  average  reduction  in  subtest  length  using  a  termination  crite¬ 
rion  of  .05  occurred  for  Subtest  10  and  amounted  to  an  average  decrease  of  67% 
of  the  items.  The  smallest  average  decrease  was  observed  for  Subtest  2,  which 
showed  a  39%  reduction.  For  the  .01  termination  rule  the  largest  reduction  oc¬ 
curred  for  Subtest  12  with  an  average  60%  reduction,  while  Subtest  1  showed  the 
smallest  average  reduction  of  27%. 

Inter-subtest  branching.  When  the  inter-subtest  branching  strategy  was 
employed  to  implement  differential  subtest  entry  in  addition  to  adaptive  intra¬ 
subtest  item  selection,  test  lengths  were  generally  reduced  even  further.  Table 
6  shows  the  mean  test  lengths  when  subtests  were  ordered  by  multiple  correla¬ 
tion,  by  the  number  of  items,  and  randomly.  However,  compared  to  the  results 
from  intra-subtest  item  selection  alone  (Table  5),  these  reductions  were  slight. 

For  example.  Table  5  shows  that  the  mean  total  test  battery  length  when 
using  only  Intra-subtest  adaptive  item  selection  with  a  termination  criterion  of 
.05  was  95.78,  or  a  52.3%  reduction.  When  coupled  with  inter-subtest  adaptive 
branching  (Table  6),  mean  test  battery  lengths  decreased  to  94.10  (53.2%  reduc¬ 
tion)  for  ordering  by  multiple  correlation,  93.67  (53.4%  reduction)  for  ordering 
by  number  of  items,  and  94.94  (52.8%  reduction)  for  random  ordering.  For  the 
.01  termination  criterion,  intra-subtest  item  selection  alone  yielded  a  mean 
test  battery  length  of  115.55,  or  a  42.5%  reduction.  When  inter-subtest  branch¬ 
ing  was  added,  mean  test  battery  lengths  of  115.29,  114.45,  and  114.57  were  ob¬ 
served,  representing  mean  test  battery  reductions  of  42.6%,  43.0%,  and  43.0% 
from  the  full  test  battery  length.  In  general  (across  all  three  ordering  proce¬ 
dures),  additional  reductions  in  test  length  due  to  inter-subtest  branching 
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Table  6 

Number  of  Items  Administered  In  12  Adaptive  Subtests  Ordered 
by  Highest  Multiple  Correlation,  by  Number  of  Items,  and  Randomly, 
Using  Differential  Subtest  Entry 


Ordering 

Termination 

Criterion 

of  .05 

Termination  Criterion 

of  .01 

Method  Conven- 

Percent 

Percent 

and  tional 

Range 

Reduc- 

Range 

Reduc- 

Subtest  Test 

Mean  SD 

Min  Max 

tion 

Mean  SD  Min  Max 

tion 

Highest  Multiple 
Correlation 


3 

15 

7.55 

1.43 

3 

11 

49.  7 

9.09 

1.50 

4 

12 

39.4 

11 

24 

11.83 

2.77 

4 

15 

50.  7 

14.42 

2.12 

7 

19 

39.9 

5 

17 

9.09 

2.26 

3 

12 

46.  5 

11.14 

2.44 

5 

14 

34.5 

7 

13 

4.90 

1.38 

1 

11 

62.  3 

5.74 

1.61 

1 

12 

55.8 

9 

22 

11.54 

2.92 

3 

17 

47.  5 

13.96 

3.11 

5 

18 

36.5 

4 

19 

10.75 

2.14 

4 

14 

43.  4 

12.88 

1.81 

5 

15 

32.2 

1 

10 

6.16 

1.24 

1 

8 

38.  4 

7.52 

.93 

3 

9 

24.8 

6 

18 

7.13 

1.85 

2 

10 

60.  4 

8.68 

2.09 

2 

14 

51.8 

2 

10 

5.87 

1.24 

2 

8 

41.  3 

6.76 

1.00 

3 

8 

32.4 

10 

23 

7.94 

4.09 

2 

21 

65.  5 

10.77 

4.84 

2 

21 

53.2 

12 

18 

5.86 

1.69 

2 

12 

67.  4 

7.18 

2.20 

2 

16 

60.1 

8 

12 

5.48 

2.13 

1 

10 

54.  3 

7.15 

2.23 

2 

11 

40.4 

Total 

201 

94.10 

28 

149 

53.  2 

115.29 

41 

169 

42.6 

Number 

of  Items 

11 

24 

11.93 

2.62 

6 

16 

50.  3 

14.50 

2.01 

8 

20 

39.6 

10 

23 

7.81 

3.92 

2 

20 

66.  0 

10.48 

4.69 

2 

21 

54.4 

9 

22 

11.51 

2.90 

5 

16 

47.  7 

13.99 

3.07 

7 

19 

36.4 

4 

19 

10.64 

2.23 

4 

14 

44.  0 

12.83 

2.04 

5 

15 

32.5 

12 

18 

5.93 

1.92 

2 

15 

67.  0 

7.23 

2.25 

2 

15 

59.8 

6 

18 

7.21 

1.91 

2 

10 

59.  9 

8.80 

1.98 

3 

14 

51.1 

5 

17 

9.03 

2.32 

3 

13 

46.  9 

10.82 

2.65 

5 

14 

36.4 

3 

15 

7.36 

1.39 

2 

11 

50.  9 

8.99 

1.70 

3 

11 

40.1 

7 

13 

4.64 

1.54 

1 

12 

64.  3 

5.64 

1.77 

1 

12 

56.6 

8 

12 

5.60 

2.22 

1 

9 

53.  3 

7.02 

2.20 

2 

10 

41.5 

2 

10 

5.90 

1.25 

2 

9 

41.  0 

6.69 

1.01 

2 

8 

33.1 

1 

10 

6.11 

1.34 

1 

8 

38.  9 

7.46 

.81 

5 

9 

25.4 

Total 

201 

93.67 

31 

153 

53.  4 

114.45 

45 

168 

43.0 

Random 

3 

15 

7.53 

1.55 

3 

11 

49.  8 

9.09 

1.50 

4 

12 

39.4 

4 

19 

10.89 

2.08 

5 

15 

42.  7 

12.89 

1.82 

5 

15 

32.1 

11 

24 

11.92 

2.81 

3 

16 

50.  3 

14.35 

2.08 

5 

20 

40.2 

2 

10 

5.91 

1.25 

2 

8 

40.  9 

6.73 

1.02 

3 

8 

32.7 

7 

13 

4.92 

1.60 

1 

12 

62.  2 

5.57 

1.72 

1 

12 

57.2 

6 

18 

7.36 

1.74 

2 

10 

59.  1 

8.83 

2.01 

3 

15 

50.9 

1 

10 

6.04 

1.36 

1 

8 

39.  6 

7.48 

.90 

4 

10 

25.2 

5 

17 

8.89 

2.34 

2 

12 

47.  7 

10.87 

2.72 

3 

15 

36.1 

8 

12 

5.72 

2.12 

1 

10 

52.  3 

7.13 

2.26 

2 

10 

40.6 

10 

23 

8.04 

4.27 

2 

21 

65.  0 

10.51 

4.63 

2 

22 

54.3 

12 

18 

6.05 

1.81 

1 

12 

66.  4 

7.10 

2.10 

3 

14 

60.6 

9 

22 

11.67 

3.70 

4 

17 

46.  9 

14.02 

3.22 

6 

18 

36.3 

Total 

201 

94.94 

27 

152 

52.  8 

114.57 

41 

171 

43.0 

♦Computed  by  the  formula  100-[(mean  number  of  items  in  adaptive  test/number 
of  items  in  conventional  test)  x  100] 
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amounted  to  less  than  1%  of  original  test  length  for  the  .05  termination  crite¬ 
rion  and  virtually  no  improvement  for  the  .01  termination  criterion. 

Analysis  of  the  percentage  reduction  data  for  individual  subtests  due  to 
the  various  order-by-termination  combinations  revealed  nearly  identical  results 
across  ordering  methods  within  each  termination  criteria.  The  only  discernible 
trend  was  for  the  more  stringent  termination  criterion  to  administer  approxi¬ 
mately  20%  more  items  per  subtest. 

Correlation  with  True  Achievement  Levels 


Table  7  presents  the  values  of  the  correlation  coefficients  of  the  Bayesian 
estimated  achievement  levels  with  the  true  achievement  levels  for  the  conven¬ 
tional  and  adaptive  tests.  Generally,  these  correlations  were  fairly  consistent 
across  both  termination  criteria  and  branching  strategies. 

Table  7 

Correlations  of  Bayesian  Achievement  Level  Estimates  with  the  True 
Achievement  Levels  for  12  Subtests  Using  Conventional  Administration, 
Adaptive  Intra-Subtest  Item  Selection  Only,  and  Intra-Subtest  Combined 
with  Three  Kinds  of  Inter-Subtest  Ordering,  at  Two  Termination  Levels 


Subtest 

Conven¬ 

tional 

Test 

Termination  Criterion 

of  .05 

Termination  Criterion 

of  .01 

Intra- 

Subtest 

Only 

Inter-Subtest  Ordering 

Intra- 

Subtest 

Only 

Inter-Subtest 

Ordering 

Multiple 

R 

No. 

Items 

Random 

Multiple 

R 

No. 

Items 

Random 

1 

.82 

.81 

.81 

.80 

.83 

.80 

.78 

.82 

.82 

2 

.81 

.78 

.77 

.82 

.79 

.83 

.79 

.80 

.81 

3 

.83 

.77 

.79 

.83 

.80 

.79 

.79 

.79 

.79 

4 

.84 

.79 

.84 

.84 

.82 

.85 

.83 

.81 

.84 

5 

.85 

.80 

.82 

.83 

.83 

.85 

.85 

.82 

.82 

6 

.79 

.76 

.79 

.81 

.78 

.84 

.79 

.80 

.79 

7 

.73 

.66 

.64 

.73 

.71 

.70 

.64 

.67 

.67 

8 

.82 

.79 

.79 

.82 

.80 

.84 

.82 

.78 

.79 

9 

.84 

.87 

.85 

.87 

.86 

.86 

.85 

.86 

.87 

10 

.79 

.73 

.77 

.78 

.77 

.81 

.76 

.79 

.75 

11 

.89 

.88 

.88 

.87 

.89 

.89 

.90 

.89 

.89 

12 

.72 

.63 

.71 

.75 

.68 

.71 

.67 

.67 

.67 

Median 

.82 

.79 

.79 

.82 

.82 

.83 

.79 

.80 

.79 

Adaptive  lntra-subtest  item  selection.  For  the  termination  criterion  of 
.05  with  no  inter-subtest  branching,  the  correlations  ranged  from  a  low  of  .63 
for  Subtest  12  to  a  high  of  .88  for  Subtest  11.  Using  a  termination  criterion 
of  .01,  the  correlations  ranged  from  a  low  of  .70  for  Subtest  7  to  a  high  of  .89 
for  Subtest  11.  The  more  stringent  termination  criterion  resulted  in  somewhat 
higher  correlations  and  reduced  the  range  of  the  observed  correlations. 

Inter-subtest  branching.  When  inter-subtest  branching  was  added  to  the 
adaptive  lntra-subtest  item  selection,  the  observed  correlations  showed  no  ap¬ 
parent  trends  with  respect  to  termination  criterion  or  ordering  procedure. 


13  - 


Although  somewhat  lower  fidelity  coefficients  were  noted  for  adaptive  ver¬ 
sus  conventional  administration,  it  should  be  remembered  that,  on  the  average, 
the  adaptively  administered  subtests  used  30%  fewer  items  to  estimate  the 
achievement  levels  of  each  simulee  than  did  the  conventionally  administered  sub¬ 
tests.  Yet  for  many  subtests,  differences  in  these  fidelity  correlations  were 
quite  small;  in  some  cases  the  fidelity  coefficients  were  even  slightly  higher 
for  the  adaptive  tests. 

Information 


Mean  information  values  are  shown  in  Table  8  for  Subtest  8  [1(0)]  at  inter¬ 
vals  of  estimated  achievement  (9)  for  the  conventional  tests,  and  for  the  adap¬ 
tive  tests  with  test  termination  criteria  of  .05  and  .01  using  both  intra-sub¬ 
test  item  selection  only  and  inter-subtest  branching,  with  subtests  ordered  by 
highest  multiple  correlations  (Subtest  8  was  the  last  test  administered  under 
the  highest  multiple  R  ordering).  These  values  are  based  on  the  mean  informa- 

Table  8 

Mean  Information  Values  [1(0)  ]  and  Number  of  Simulees  (N)  at  Intervals 
of  Estimated  Achievement  Levels  (6 )  for  Subtest  8  Using  Termination 
Criteria  of  .05  and  .01  with  Intra-Subtest  Item  Selection  Only 
and  Combined  with  Inter-Subtest  Branching 


Conventional 

Adaptive  Intra-Subtest 
Item  Selection: 
Termination  Criterion 

Adaptive  Intra-Subtest 
Item  Selection  with 
Inter-Subtest  Branching: 
Termination  Criterion 

0  Range 

Test 

.05 

.01 

.05 

.01 

Lo 

Hi 

N 

1(0) 

N 

1(g) 

N 

K9) 

N 

1(9) 

N 

K9) 

-2.00 

-1.80 

- 

- 

— 

— 

- 

- 

- 

- 

- 

- 

-1.79 

-1.60 

- 

- 

- 

- 

- 

- 

1 

.02 

3 

.02 

-1.59 

-1.40 

6 

.09 

6 

.08 

5 

.08 

6 

.06 

8 

.12 

-1.39 

-1.20 

29 

.73 

19 

.65 

18 

.54 

7 

.66 

11 

.64 

-1.19 

-1.00 

14 

2.06 

5 

2.23 

7 

2.21 

16 

2.29 

11 

2.07 

-.99 

-.80 

- 

- 

1 

3.82 

4 

3.92 

1 

3.88 

4 

4.10 

-.79 

-.60 

22 

3.38 

- 

- 

- 

- 

14 

3.33 

11 

3.28 

-.59 

-.40 

24 

2.30 

49 

2.18 

48 

2.44 

52 

1.98 

33 

2.08 

-.39 

-.20 

36 

1.54 

49 

1.48 

37 

1.54 

30 

1.46 

37 

1.54 

-.19 

.00 

41 

1.76 

20 

1.67 

38 

1.72 

33 

1.66 

39 

1.77 

.01 

.20 

22 

2.35 

41 

2.18 

31 

2.26 

31 

2.29 

29 

2.26 

.21 

.40 

24 

3.43 

25 

2.98 

26 

3.16 

29 

3.24 

25 

3.25 

.41 

.60 

19 

4.52 

30 

4.36 

24 

4.42 

20 

4.48 

28 

4.46 

.61 

.80 

12 

4.97 

12 

4.92 

9 

4.96 

16 

4.91 

11 

4.96 

.81 

1.00 

8 

4.70 

4 

4.83 

16 

4.76 

10 

4.75 

12 

4.75 

1.01 

1.20 

14 

4.65 

10 

4.74 

18 

4.75 

9 

4.69 

14 

4.65 

1.21 

1.40 

6 

5.25 

2 

4.92 

3 

4.92 

6 

5.10 

9 

5.10 

1.41 

1.60 

15 

5.31 

6 

5.33 

4 

5.34 

5 

5.26 

5 

5.18 

1.61 

1.80 

2 

4.55 

10 

4.68 

7 

4.68 

4 

4.82 

2 

4.32 

1.81 

2.00 

8 

2.96 

11 

2.11 

5 

3.21 

10 

2.61 

8 

2.57 

Total 

Group 

300 

2.75 

300 

2.58 

300 

2.77 

300 

2.73 

300 

2.76 

tion  in  the  test  items  actually  administered  to  each  simulee,  using  its  6  at  the 
termination  of  each  subtest.  The  differences  in  mean  information  between  the 
two  termination  conditions  tended  to  be  small,  and  no  trends  were  apparent.  The 
strong  similarities  in  the  data  for  individual  subtests  resulting  from  the  two 
termination  criteria  suggested  that  little  was  to  be  gained  by  use  of  the  more 
stringent  .01  termination  criterion.  Thus,  the  remaining  analyses  were  conduc¬ 
ted  using  only  the  .05  criterion.  Appendix  Tables  E  through  P  present  the  mean 
values  of  estimated  information  at  intervals  of  0  for  the  conventional  and  adap¬ 
tive  subtests  under  all  test  conditions  for  the  termination  criterion  of  .05. 

Adaptive  intra-subtest  item  selection.  Based  on  the  data  in  Appendix  Ta¬ 
bles  E  through  P  for  the  conventional  test  and  the  adaptive  test  with  adaptive 
inter-subtest  item  selection  only,  Table  9  presents  the  means  of  the  average 
subtest  information  values  across  levels  of  §,  their  coefficients  of  variation, 
and  their  efficiencies  relative  to  the  conventional  test.  The  mean  values  of 
subtest  information  for  each  subtest  were  virtually  identical  for  the  adaptive 
test  compared  to  the  conventional  test.  The  coefficients  of  variation  for  the 
adaptive  subtests  also  were  quite  similar  to  the  full  length  conventional  sub¬ 
tests.  The  efficiency  of  adaptive  intra-subtest  item  selection  relative  to  the 
full  length  conventional  subtests  appear  in  the  last  column  of  Table  9.  These 
data  suggest  that  there  was  little,  if  any,  loss  of  information  incurred  by  the 
adaptive  strategy,  even  though  the  adaptive  subtests  averaged  50%  fewer  items. 

Table  9 

Mean  of  Average  Test  Information,  Coefficient  of  Variation  (CV), 

Range  of  Mean  Test  Information  Across  0  Levels,  and 
Relative  Mean  Efficiency  to  Conventional  Test  for  Adaptive  Tests  with 
Intra-Subtest  Item  Selection  Only  and  Termination  Criterion  of  .05 


Subtest 

Conventional  Test 

Adaptive  Test 

Range 

Relative 

Mean 

Mean 

CV 

Mean 

CV 

Min 

Max 

Efficiency 

1 

2.75 

.57 

2.73 

.64 

.25 

6.43 

.99 

2 

2.35 

.22 

2.46 

.35 

1.66 

6.78 

1.05 

3 

2.72 

.29 

2.76 

.30 

.46 

4.30 

1.01 

4 

3.25 

.14 

3.17 

.16 

.80 

5.10 

.98 

5 

3.22 

.26 

3.09 

.29 

.65 

4.31 

.96 

6 

2.17 

.18 

2.08 

.24 

.24 

2.62 

.96 

7 

1.25 

.61 

1.19 

.56 

.44 

5.03 

.95 

8 

2.75 

.53 

2.58 

.52 

.08 

5.34 

.94 

9 

4.26 

.22 

4.23 

.24 

.72 

6.86 

.99 

10 

2.46 

.51 

2.40 

.52 

.34 

12.04 

.98 

11 

5.50 

.43 

5.55 

.44 

.54 

8.61 

1.01 

12 

1.75 

.48 

1.81 

.44 

.05 

6.52 

1.03 

Total 

Battery 

34.43 

34.05 

.99 

Figure  1  graphically  depicts  the  overall  test  Information  curves  obtained 
when  the  tests  were  administered  conventionally,  with  only  adaptive  intra-sub¬ 
test  item  selection,  and  with  inter-subtest  branching  in  addition  to  intra-sub¬ 
test  item  selection.  Whereas  Table  9  showed  average  values  of  information  over 
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achievement  levels.  Figure  1  shows  the  mean  of  the  average  information  values 
over  subtests  as  a  function  of  achievement  level.  The  curves  are  very  similar, 
with  the  largest  differences  occurring  at  the  6  interval  -1.59  to  -1.40,  where 
mean  information  of  the  adaptive  lntra-subtest  strategy  was  14.66  and  that  of 
the  conventional  test  was  7.82;  and  at  the  §  intervals  .61  to  .80,  .81  to  1.00, 
1.01  to  1.20,  and  1.41  to  1.60  where  the  conventional  test's  information  values 
were  37.61,  32.01,  29.10,  and  26.54,  respectively,  and  those  of  the  adaptive 
test  using  only  adaptive  intra-subtest  item  selection  were  30.13,  25.51,  22.10, 
and  20.04,  respectively. 


Figure  1 

Mean  Test  Information  Curves  Across  12  Subtests  for  Conventional  Test 
and  for  Adaptive  Test  Using  Intra-Subtest  Item  Selection  Only 
and  Combined  with  Inter-Subtest  Branching 


Inter-subtest  branching.  Table  10  presents  the  mean  subtest  information 
values,  their  coefficients  of  variation,  and  efficiencies  relative  to  the  con¬ 
ventionally  administered  subtests  for  the  adaptive  tests  using  three  methods  of 
ordering  subtests  for  inter-subtest  branching.  Comparison  of  the  data  in  Table 
10  with  that  of  Table  9  shows  that  the  addition  of  inter-subtest  branching  to 
the  adaptive  intra-subtest  item  selection  appeared  to  have  a  minimal  effect  on 
these  evaluative  criteria.  Compared  to  the  conventional  test,  the  adaptive  test 
with  inter-subtest  branching  showed  the  same  pattern  of  overall  subtest  mean 
information,  as  reflected  in  relative  mean  efficiencies  very  close  to  1.00  for 
all  subtests  and  branching  conditions. 


Figure  1  also  shows  the  average  information  curve  across  the  12  subtests 
for  the  inter-subtest  branching  strategy  using  highest  multiple  correlations. 
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Table  10 

Mean  of  Average  Test  Information,  Coefficient  of  Variation  (CV), 
Range  of  Mean  Test  Information  Across  6  Levels,  and  Relative  Mean 
Efficiency  to  Conventional  Test  for  Adaptive  Tests  with  Three  Types 
of  Inter-Subtest  Branching  and  Termination  Criterion  of  .05 


Ordering 

Method 

and 

Subtest 

Conventional 

Test 

Mean 

Adapti 

CV 

ve  Test 

Range 

Relative 

Mean 

Efficiency 

Mean 

CV 

Min 

Max 

Highest  Multiple 

Correlation 

3 

2.72 

.29 

2.76 

.30 

.46 

4.34 

1.01 

11 

5.50 

.43 

5.29 

.46 

.40 

8.61 

.96 

5 

3.22 

.26 

3.08 

.29 

.43 

4.28 

.96 

7 

1  ,25 

.61 

1.31 

.68 

.12 

9.51 

1.05 

9 

*.26 

.22 

4.33 

.22 

.21 

6.58 

1.02 

4 

3.25 

.14 

3.15 

.16 

.44 

7.07 

.97 

1 

2.75 

.57 

2.71 

.58 

.13 

6.46 

.98 

6 

2.17 

.18 

2.11 

.23 

.04 

2.63 

.97 

2 

2.35 

.22 

2.44 

.32 

1.13 

6.78 

1.04 

10 

2.46 

.51 

2.53 

.61 

.14 

12.16 

1.03 

12 

1.75 

.48 

1.84 

.47 

.10 

7.86 

1.05 

8 

2.75 

.53 

2.73 

.49 

.02 

5.36 

.99 

Total  Battery 

34.43 

34.28 

.99 

Number  of  Items 

11 

5.50 

.43 

5.28 

.45 

.81 

8.60 

.96 

10 

2.46 

.51 

2.33 

.49 

.09 

8.34 

.95 

9 

4.26 

.22 

4.24 

.25 

.59 

6.96 

.99 

4 

3.25 

.14 

3.16 

.17 

.63 

7.35 

.97 

12 

1.75 

.48 

1.83 

.53 

.04 

8.17 

1.04 

6 

2.17 

.18 

2.11 

.21 

.13 

2.63 

.97 

5 

3.22 

.26 

3.05 

.30 

.50 

4.33 

.95 

3 

2.72 

.29 

2.72 

.28 

.14 

4.29 

1.00 

7 

1.25 

.61 

1.28 

.78 

.11 

14.52 

1.02 

8 

2.75 

.53 

2.76 

.51 

.02 

5.30 

1.00 

2 

2.35 

.22 

2.40 

.27 

1.14 

6.66 

1.02 

1 

2.75 

.57 

2.73 

.61 

.08 

6.44 

.99 

Total  Battery 

34.43 

33.89 

.98 

Random 

3 

2.72 

.29 

2.76 

.32 

.46 

4.34 

1.01 

4 

3.25 

.14 

3.17 

.12 

.86 

5.54 

.98 

11 

5.50 

.43 

5.55 

.43 

.15 

8.61 

1.01 

2 

7.35 

.22 

2.41 

.31 

1.14 

6.77 

1.02 

7 

1.25 

.61 

1.41 

1.12 

.06 

14.60 

1.13 

6 

2.17 

.18 

2.15 

.20 

.08 

2.64 

.99 

1 

2.75 

.57 

2.71 

.59 

.11 

6.41 

.98 

5 

3.22 

.26 

3.03 

.28 

.35 

4.25 

.94 

8 

2.75 

.53 

2.70 

.54 

.02 

5.36 

.98 

10 

2.46 

.51 

2.42 

.55 

.27 

11.97 

.98 

12 

1.75 

.48 

1.85 

.45 

.03 

6.30 

1.06 

9 

4.26 

.22 

4.32 

.24 

.36 

5.67 

1.01 

Total  Battery 

34.43 

34.48 

1.00 

- 
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Here,  again,  there  is  very  little  separation  between  the  conventional  test's 
curve  and  the  curve  for  inter-subtest  branching.  One  effect  of  intra-subtest 
item  selection  combined  with  inter-subtest  branching  over  intra-subtest  item 
selection  alone  was  to^  narrow  the  gap  between  the  conventional  test  and  the 
adaptive  test  in  the  0  intervals  where  intra-subtest  item  selection  alone  fell 
below  the  conventional  test.  Similar  results  were  obtained  when  subtests  were 
ordered  by  number  of  items  and  randomly  for  use  in  the  adaptive  inter-subtest 
branching  procedure. 


DISCUSSION 

This  study  has  replicated  previous  findings  (Brown  &  Weiss,  1977;  Gialluca 
&  Weiss,  1979)  that  showed  that  an  adaptive  testing  strategy  combining  intra¬ 
subtest  adaptive  item  selection  with  inter-subtest  branching  could  significantly 
reduce  the  length  of  an  achievement  test  battery  while  maintaining  desirable 
psychometric  properties.  The  present  study  applied  this  test  design  to  the  same 
basic  data  set  used  by  Brown  and  Weiss  (1977)  but  used  a  monte  carlo  simulation 
to  assess  the  separate  effects  of  adaptive  intra-subtest  item  selection  and  in¬ 
ter-subtest  branching  on  test  length,  test  information,  and  score  fidelity.  In 
addition,  this  study  investigated  the  effects  of  various  subtest  orderings  for 
inter-subtest  branching  on  the  psychometric  properties  of  the  subtests. 

Adaptive  Intra-Subtest  Item  Selection  f 

The  adaptive  intra-subtest  item  selection  strategy  used  in  this  study  was 
identical  to  that  used  by  Brown  and  Weiss  (1977)  and  Gialluca  and  Weiss  (1979) 
in  that  items  were  selected  for  subsequent  administration  by  selecting  the  item 
that  provided  the  highest  amount  of  psychometric  information  available  at  the 
current  estimate  of  0.  Although  such  maximum  information  item  selection  is  typ¬ 
ically  used  with  maximum  likelihood  scoring,  the  present  application  used  a 
Bayesian  scoring  method  for  three  reasons: 

1.  Maximum  likelihood  scoring  requires  that  an  examinee's  response  vector 
contain  at  least  one  correct  and  one  incorrect  response  before  a  0  esti¬ 
mate  can  be  computed; 

2.  Maximum  likelihood  scoring  does  not  use  prior  information  as  effectively 
as  does  Bayesian  scoring;  and 

3.  Some  response  patterns  result  in  a  failure  of  the  maximum  likelihood  pro¬ 
cedure  to  converge  upon  a  0  estimate. 

In  general,  although  scores  obtained  from  Bayesian  and  maximum  likelihood 
methods  tend  to  be  highly  correlated,  they  do  not  yield  numerically  identical 
results  given  the  same  data.  The  extent  to  which  these  two  scoring  procedures 
yield  numerically  discrepant  results  was  reported  by  Kingsbury  and  Weiss  (1979). 

Unfortunately,  the  issue  of  the  most  appropriate  choice  of  scoring  algorithm 
pervades  implementations  of  IRT-based  testing  in  general  and  hence  should  be 
addressed  in  future  research. 


In  terms  of  fidelity  or  correlations  with  true  achievement  levels,  the 
adaptive  intra-subtest  item  selection  strategy  using  the  .05  termination  crite- 


rloni  resulted  in  lower  validity  coefficients  than  the  conventionally  adminis¬ 
tered  subtests.  This  was  likely  attributable,  in  part,  to  the  combined  effects 
of  the  Bayesian  regressed  achievement  estimate  and  the  smaller  number  of  items 
used  for  the  estimates.  This,  in  turn,  caused  a  restriction  in  the  range  of 
estimated  achievement  levels,  thus  lowering  the  correlation  with  true  achieve¬ 
ment  levels.  The  same  pattern,  however,  did  not  appear  when  the  more  stringent 
.01  termination  criterion  was  used,  further  supporting  the  hypothesized  effects 
of  a  restricted  range  of  Bayesian  achievement  level  estimates. 

Test  length  reductions  of  39%  to  67%  resulted  from  the  variable  termination 
criterion  of  the  intra-subtest  item  selection  strategy.  In  spite  of  the  consid¬ 
erable  decrease  in  test  length,  there  was  virtually  no  loss  in  the  amount  of 
psychometric  information  obtained  from  each  subtest. 

Inter-Subtest  Branching 


The  use  of  prior  information  to  determine  differential  subtest  entry  into 
subsequent  subtests  appeared  to  have  little  effect  in  terms  of  number  of  items 
administered  over  and  above  that  produced  by  intra-subtest  item  selection  alone. 
The  extent  to  which  this  finding  is  generallzable  across  subtests  with  different 
characteristics  and  different  intercorrelations  remains  to  be  investigated  in 
future  research. 

The  addition  of  inter-subtest  branching  to  the  adaptive  intra-subtest  item 
selection  generally  raised  the  low  validities  obtained  under  the  .05  termination 
condition  using  only  adaptive  intra-subtest  item  selection  to  near  those  ob¬ 
tained  with  conventional  administration.  This  same  effect  appeared  using  the 
.01  termination  criterion  but  to  a  lesser  extent. 

A  possible  explanation  for  the  small  effects  of  prior  information  may  lie 
in  the  combination  of  the  use  of  linear  multiple  regression  estimates  to  deter¬ 
mine  prior  ability  estimates,  coupled  with  a  regressed  Bayesian  scoring  algo¬ 
rithm.  That  is,  the  regression  estimates  themselves  tend  to  underestimate  or 
regress  extreme  scores  and,  together  with  the  tendency  of  the  Bayesian  procedure 
to  further  regress  the  achievement  estimates,  might  tend  to  offset  any  benefit 
arising  from  prior  information.  Perhaps  the  benefit  of  inter-subtest  branching 
is  not  to  further  reduce  test  length  but  to  maintain  the  original  range  of 
achievement  levels  against  the  tendency  of  Bayesian  estimation  to  regress  them. 
Maximum  likelihood  scoring  might  also  be  useful  at  the  end  of  each  subtest  to 
obtain  unregressed  ability  estimates  for  use  in  inter-subtest  adaptive  branch¬ 
ing. 


Consistent  with  the  results  of  test  length  reduction,  mean  subtest  informa¬ 
tion  was  not  significantly  changed  when  inter-subtest  branching  was  added  to  the 
adaptive  intra-subtest  item  selection  strategy.  Apparently,  the  regression- 
based  achievement  estimates  had  the  effect  of  selecting  the  same  or  similar 
items  as  those  selected  by  the  testing  procedure  that  did  not  use  differential 
subtest  entry.  This  may  have  been  caused  by  the  rather  low  to  moderate  inter¬ 
correlations  used  in  the  simulation.  For  example,  the  range  of  intercorrela¬ 
tions  used  for  the  present  study  ranged  from  .14  to  .53,  with  a  mean  of  .34. 
Different  results  might  be  obtained  with  more  highly  correlated  tests,  such  as 
those  in  the  Armed  Services  Vocational  Aptitude  Battery;  ASVAB-9  intercorrela¬ 
tions  range  from  .12  to  .75  with  a  mean  of  .50. 


Conclusions 


The  results  of  this  study  replicated  and  extended  the  findings  reported  by 
Brown  and  Weiss  (1977)  and  Gialluca  and  Weiss  (1979).  That  is,  test  lengths 
could  be  reduced  by  39%  to  67%  using  the  Brown  and  Weiss  (1977)  adaptive  testing 
strategy  designed  for  test  batteries. 

The  design  of  the  present  study  allowed  the  separation  of  the  effects  due 
to  adaptive  intra-subtest  item  selection  from  that  due  to  inter-subtest  branch¬ 
ing.  Although  the  results  showed  that  the  intra-subtest  item  selection  account¬ 
ed  for  virtually  all  the  reduction  in  test  length,  the  addition  of  prior  infor¬ 
mation  using  inter-subtest  branching  appeared  to  have  an  effect  on  the  observed 
validities  and  on  mean  battery  information.  Achievement  level  estimates  ob¬ 
tained  using  inter-subtest  branching  combined  with  adaptive  intra-subtest  item 
selection  showed  validities  quite  comparable  to  those  of  conventional  tests 
nearly  twice  as  long.  As  observed  in  previous  studies,  there  was  a  minimal  loss 
of  psychometric  information  due  to  the  adaptive  procedure. 

Although  this  study  replicated  and  extended  the  findings  of  the  previous 
studies,  it,  too,  was  limited  by  the  fact  that  only  a  small  number  of  factors 
were  included.  The  next  step  should  be  a  large-scale  simulation  that  systemati¬ 
cally  varies  the  important  sources  of  variation. 

Future  research  on  inter-subtest  branching  should  be  concerned  with  the 
determination  of  the  conditions  under  which  it  is  maximally  effective.  The  fac¬ 
tors  examined  should  include  (1)  varying  the  factorial  composition  of  the  sub¬ 
tests  as  a  way  of  systematically  generating  realistic  correlation  matrices;  (2) 
comparing  the  effects  of  different  scoring  algorithms — such  as  Owen's  Bayesian, 
modal  Bayesian  (which  differs  from  Owen's  in  using  the  mode  of  the  posterior 
distribution  rather  than  the  mean),  and  maximum  likelihood  coupled  with  Bayesian 
scoring  (to  eliminate  the  effects  of  regressed  Bayesian  estimates);  (3)  various 
subtest  orderings;  and  (4)  effects  as  a  function  of  achievement  level  as  well  as 
averaged  over  the  levels.  A  double  cross-validation  design  such  as  used  by 
Gialluca  and  Weiss  (1979)  appears  to  be  quite  useful  in  this  regard. 

Test  length  in  adaptive  testing  using  a  variable  termination  criterion  is  a 
function  of  the  magnitude  of  the  termination  criterion  used.  The  termination 
criterion  that  is  optimal  for  a  given  situation — the  magnitude  of  information 
that  can  be  maintained  while  eliminating  non-inf orraative  items — also  needs  to  be 
addressed  in  future  studies.  The  most  important  factor  in  this  regard  is  the 
information  structure  of  the  subtests.  Thus,  a  complete  factorial  design  should 
also  include  manipulation  of  the  subtest  information  curves,  based  on  informa¬ 
tion  curves  observed  in  other  test  batteries  as  well  as  on  ideal  information 
curves  for  use  in  adaptive  testing  batteries. 
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Note .  Dashed  lines  indicate  that  an  item  was  rejected  in  the  first  phase  of  the  Item  parameterization  procedure. 
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Table  B 

Raw  Score  Regression  Weights,  Regression  Constants,  and 
Squared  Standard  Errors  of  Estimate  (SSEE)  for  Regression 
Equations  Used  to  Determine  Differential  Entry  Points, 
with  Subtests  Ordered  by  Highest  R 


Subtest 


Subtest 

3 

11 

5 

7 

9 

6 

1 

6 

2 

10 

12  8 

u 

562 

-Oil 

5 

396 

178 

019 

7 

177 

176 

315 

063 

9 

116 

079 

266 

215 

-017 

4 

201 

123 

073 

055 

269 

-061 

1 

163 

201 

130 

056 

-076 

195 

-014 

6 

085 

-068 

052 

121 

300 

143 

080 

-015 

2 

on 

021 

168 

178 

088 

166 

106 

-008 

047 

10 

015 

035 

026 

102 

143 

052 

-045 

111 

222 

-006 

12 

-051 

180 

101 

038 

023 

-131 

108 

-004 

167 

187 

-Oil 

8 

066 

008 

107 

135 

013 

025 

021 

072 

071 

064 

121  -001 

SSEE 

755 

740 

691 

632 

668 

716 

745 

764 

766 

718 

772 

Note .  Regression  constants  appear  on  the  main  diagonal. 


Table  C 

Raw  Score  Regression  Weights,  Regression  Constants,  and 
Squared  Standard  Errors  of  Estimate  (SSEE)  for  Regression 
Equations  Used  to  Determine  Differential  Entry  Points, 


with  Subtests  Ordered  by  Number  of  Items 


Subtest 

Subtest 

ii 

10 

9 

* 

12 

6 

5 

3 

7 

8 

2  1 

10 

220 

006 

9 

253 

273 

-006 

4 

225 

111 

309 

-040 

12 

241 

242 

081 

-065 

002 

6 

010 

121 

335 

186 

019 

-010 

5 

168 

048 

285 

120 

135 

094 

022 

3 

318 

024 

049 

151 

-016 

078 

223 

-004 

7 

125 

099 

152 

031 

059 

094 

206 

117 

046 

8 

012 

075 

014 

042 

136 

072 

119 

050 

146 

002 

2 

-002 

166 

039 

177 

167 

-028 

117 

026 

132 

067 

046 

1 

180 

-062 

-010 

178 

103 

079 

096 

156 

024 

019 

087  -016 

SSEE 

924 

729 

697 

765 

762 

673 

538 

638 

774 

712 

698 

Note*  Regression  constants  appear  on  the  main  diagonal. 


Table  D 

Raw  Score  Regression  Weights,  Regression  Constants, 
and  Squared  Standard  Errors  of  Estimate  (SSEE)  for  Regression 
Equations  Used  to  Determine  Differential  Entry  Points, 
with  Subtests  Ordered  Randomly 


Subtest 

Subtest 

■  i  »  '  c 

3 

4 

11 

2 

7 

6 

1 

5 

8 

10 

12  9 

4 

425 

-040 

11 

482 

190 

-003 

2 

143 

263 

112 

061 

7 

219 

098 

175 

222 

041 

6 

148 

233 

-029 

036 

210 

-019 

1 

170 

155 

203 

105 

032 

062 

-016 

5 

226 

061 

065 

137 

224 

097 

079 

006 

8 

041 

018 

035 

113 

152 

087 

028 

128 

-003 

10 

018 

075 

042 

222 

109 

-135 

-059 

044 

087 

-006 

12 

-054 

-128 

178 

158 

026 

-008 

103 

092 

112 

179 

-Oil 

9 

046 

169 

061 

036 

127 

208 

-078 

195 

009 

098 

017  -006 

SSEE 

763 

726 

840 

706 

810 

715 

629 

786 

771 

709 

547 

Note.  Regression  constants  appear  on  the  main  diagonal. 


Table  E 

Mean  Information  Values  (I)  at  Estimated  Achievement  Levels  (§) 
for  Subtest  1  under  All  Testing  Conditions 


Conven¬ 

tional 

§  Range  Test 

Lo  Hi  N  1(g) 


Intra- 

Subtest 

Item 

Selection 


N  1(0) 


Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R _  Items  Random 

15  I(T)  ~  1(f)  N  1(6) 


-2.00 

-1.80 

- 

- 

_ 

- 

_ 

_ 

-1.79 

-1.60 

2 

.23 

- 

- 

1 

.12 

5 

.14 

3 

.13 

-1.59 

-1.40 

11 

.58 

10 

.54 

12 

.58 

8 

.55 

8 

.41 

-1.39 

-1.20 

12 

1.35 

15 

1.34 

9 

1.49 

14 

1.49 

19 

1.41 

-1.19 

-1.00 

22 

2.50 

15 

2.66 

17 

2.65 

15 

2.61 

16 

2.60 

-.99 

-.80 

24 

3.79 

23 

3.91 

20 

3.60 

15 

3.71 

24 

3.84 

-.79 

-.60 

13 

4.94 

14 

5.39 

13 

5.48 

18 

5.48 

14 

5.04 

-.59 

-.40 

11 

6.28 

27 

6.25 

18 

6.28 

18 

6.28 

16 

6.27 

-.39 

-.20 

15 

5.78 

5 

5.58 

13 

5.49 

11 

5.57 

12 

5.78 

-.19 

.00 

27 

4.18 

27 

4.14 

16 

3.95 

22 

4.22 

25 

4.09 

.01 

.20 

40 

3.26 

26 

3.12 

32 

3.12 

31 

3.11 

21 

3.08 

.21 

.40 

8 

2.49 

- 

- 

18 

2.59 

20 

2.58 

20 

2.59 

.41 

.60 

27 

2.21 

28 

2.31 

42 

2.11 

28 

2.11 

28 

2.09 

.61 

.80 

30 

1.67 

110 

1.22 

46 

1.62 

50 

1.59 

52 

1.60 

.81 

1.00 

12 

1.41 

- 

- 

29 

1.20 

29 

1.18 

32 

1.20 

1.01 

1.20 

46 

1.00 

- 

- 

9 

.87 

11 

.81 

6 

.83 

1.21 

1.40 

- 

- 

- 

- 

5 

.75 

3 

.72 

1 

.72 

1.41 

1.60 

- 

- 

- 

- 

- 

- 

2 

.78 

2 

.80 

1.61 

1.80 

- 

- 

- 

- 

- 

- 

- 

- 

1.81 

2.00 

- 

- 

- 

- 

- 

- 

- 

- 

Total  Group 

300 

2.75 

300 

2.73 

300 

2.71 

300 

2.73 

300 

2.71 

Table  F 

Mean  Information  Values  (I)  at  Estimated  Achievement 
for  Subtest  2  under  All  Testing  Conditions 

Levels 

(e) 

0  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

Hi 

N 

1(9) 

N 

1(0) 

N 

l(S) 

N 

1(8) 

N 

1(0) 

-2.00 

-1.80 

7 

3.37 

_ 

_ 

5 

3.52 

5 

3.25 

7 

2.80 

-1.79 

-1.60 

3 

4.44 

14 

5.28 

4 

6.70 

2 

5.62 

5 

6.08 

-1.59 

-1.40 

- 

- 

2 

6.75 

2 

6.12 

2 

6.59 

2 

6.77 

-1.39 

-1.20 

9 

2.47 

11 

2.13 

5 

2.94 

6 

2.82 

4 

2.63 

-1.19 

-1.00 

21 

2.01 

25 

2.21 

9 

1.97 

10 

1.99 

4 

1.97 

-.99 

-.80 

8 

2.20 

7 

2.55 

18 

2.18 

21 

2.19 

25 

2.16 

-.79 

-.60 

14 

2.60 

26 

2.53 

19 

2.56 

23 

2.61 

18 

2.53 

-.59 

-.40 

29 

2.57 

43 

2.38 

32 

2.56 

18 

2.54 

27 

2.54 

-.39 

-.20 

25 

2.40 

18 

2.52 

22 

2.39 

28 

2.38 

27 

2.38 

-.19 

.00 

37 

2.48 

37 

2.79 

25 

2.44 

28 

2.45 

22 

2.47 

.01 

.20 

17 

2.71 

13 

2.90 

35 

2.74 

30 

2.75 

21 

2.78 

.21 

.40 

23 

2.89 

24 

2.84 

24 

2.89 

28 

2.89 

28 

2.89 

.41 

.60 

28 

2.69 

- 

- 

18 

2.76 

15 

2.69 

19 

2.77 

.61 

.80 

20 

2.13 

- 

- 

18 

2.32 

20 

2.33 

21 

2.28 

.81 

1.00 

- 

- 

80 

1.66 

32 

1.84 

37 

1.84 

41 

1.82 

1.01 

1.20 

59 

1.56 

- 

- 

17 

1.49 

14 

1.46 

23 

1.46 

1.21 

1.40 

- 

- 

- 

- 

12 

1.16 

9 

1.16 

5 

1.16 

1.41 

1.60 

- 

- 

- 

- 

3 

1.23 

4 

1.26 

1 

1.37 

1.61 

1.80 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

1.81 

2.00 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Total 

Group 

300 

2.35 

300 

2.46 

300 

2.44 

300 

2.40 

300 

2.41 

4 


i 

-  24  - 


Table  G 

Mean  Information  Values  (1)  at  Estimated  Achievement  Levels  (g) 
for  Subtest  3  under  All  Testing  Conditions 


Conven¬ 

tional 

g  Range  Test 

Lo  Hi  N  1(e) 


Intra- 

Subtest 

Item 

Selection 

15  ITS) 


Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R _  Items  Random 

~N  1(1)  N  T(l)  N  1(0) 


-2.00 

-1.80 

- 

- 

_ 

_ 

_ 

_ 

1 

.14 

_ 

-1.79 

-1.60 

4 

.78 

3 

.66 

3 

.66 

2 

.92 

5 

.70 

-1.59 

-1.40 

10 

1.86 

8 

1.76 

5 

1.58 

5 

1.69 

5 

1.37 

-1.39 

-1.20 

3 

2.85 

12 

2.78 

10 

2.72 

9 

2.68 

15 

2.75 

-1.19 

-1.00 

17 

2.87 

11 

2.81 

15 

2.81 

6 

2.87 

14 

2.82 

-.99 

-.80 

23 

2.89 

13 

2.83 

17 

2.78 

28 

2.84 

16 

2.82 

-.79 

-.60 

28 

3.46 

33 

3.35 

33 

3.30 

26 

3.31 

31 

3.35 

-.59 

-.40 

16 

4.24 

9 

4.14 

8 

4.04 

14 

4.03 

6 

4.04 

-.39 

-.20 

10 

4.24 

27 

4.16 

34 

4.21 

20 

4.17 

34 

4.21 

-.19 

.00 

25 

3.72 

21 

3.76 

14 

3.76 

20 

3.62 

27 

3.70 

.01 

.20 

23 

3.04 

41 

2.95 

36 

2.94 

35 

3.00 

26 

3.02 

.21 

.40 

36 

2.57 

22 

2.46 

22 

2.48 

41 

2.48 

13 

2.48 

.41 

.60 

29 

2.21 

34 

2.18 

38 

2.18 

31 

2.19 

34 

2.19 

.61 

.80 

32 

1.98 

34 

1.90 

35 

1.90 

29 

1.88 

34 

1.88 

.81 

1.00 

15 

1.81 

- 

- 

- 

- 

13 

1.70 

0 

- 

1.01 

1.20 

13 

1.75 

- 

- 

- 

- 

9 

1.67 

- 

- 

1.21 

1.40 

11 

1.84 

32 

1.78 

30 

1.76 

5 

1.82 

40 

1.78 

1.41 

1.60 

5 

2.13 

- 

- 

- 

- 

5 

2.04 

- 

- 

1.61 

1.80 

- 

- 

- 

- 

- 

- 

1 

2.35 

- 

- 

1.81 

2.00 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Total 

Croup 

300 

2.72 

300 

2.76 

300 

2.76 

300 

2.72 

300 

2.76 

Table  H 

Mean  Information  Values  (I)  at  Estimated  Achievement  Levels  (g) 
for  Subtest  4  under  All  Testing  Conditions 


g  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

Hi 

N 

I<§) 

N 

K§) 

N 

I(§) 

N 

1(8) 

N 

1(0) 

-2.00 

-1.80 

1 

.62 

_ 

_ 

1 

.44 

_ 

_ 

_ 

_ 

-1.79 

-1.60 

- 

- 

1 

.80 

2 

.76 

5 

.96 

1 

.86 

-1.59 

-1.40 

5 

1.70 

12 

1.62 

7 

1.83 

5 

1.70 

7 

1.80 

-1.39 

-1.20 

11 

2.51 

11 

2.52 

11 

2.50 

14 

2.52 

11 

2.58 

-1.19 

-1.00 

19 

3.19 

13 

3.12 

9 

3.14 

14 

3.08 

16 

3.13 

-.99 

-.80 

13 

3.37 

16 

3.31 

13 

3.30 

20 

3.31 

14 

3.30 

-.79 

-.60 

22 

3.30 

24 

3.22 

30 

3.20 

20 

3.20 

24 

3.20 

-.59 

-.40 

33 

3.28 

33 

3.23 

31 

3.23 

40 

3.24 

32 

3.22 

-.39 

-.20 

32 

3.41 

14 

3.42 

33 

3.42 

17 

3.42 

25 

3.42 

-.19 

.00 

34 

3.52 

35 

3.52 

25 

3.52 

29 

3.52 

30 

3.52 

.01 

.20 

28 

3.51 

35 

3.45 

26 

3.47 

34 

3.48 

39 

3.48 

.21 

.40 

16 

3.41 

28 

3.31 

34 

3.30 

34 

3.30 

23 

3.29 

.41 

.60 

17 

3.30 

3 

3.19 

14 

3.18 

19 

3.20 

17 

3.18 

.61 

.80 

17 

3.20 

34 

3.10 

19 

3.09 

13 

3.11 

18 

3.09 

.81 

1.00 

17 

3.09 

1 

3.09 

10 

3.04 

13 

3.03 

11 

3.03 

1.01 

1.20 

8 

2.94 

17 

2.83 

2 

2.92 

13 

3.85 

10 

3.88 

1.21 

1.40 

12 

2.71 

11 

2.66 

22 

2.64 

14 

2.62 

20 

2.61 

1.41 

1.60 

5 

2.40 

4 

2.38 

4 

2.37 

3 

2.30 

1 

2.32 

1.61 

1.80 

6 

2.90 

2 

2.67 

- 

- 

- 

- 

- 

- 

1.81 

2.00 

4 

5.46 

6 

5.03 

1 

7.07 

3 

6.24 

1 

5.54 

Total 

Group 

300 

3.25 

300 

3.17 

300 

3.15 

300 

3.16 

300 

3.17 

25 


Table  I 

Mean  Information  Values  (I)  at  Estimated  Achievement  Levels  (§) 
for  Subtest  5  under  All  Testing  Conditions 


§  Range 

Conven¬ 

tional 

Test 

Intra- 
. Subtest 
Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

HI 

N 

I(§) 

N 

K§) 

N 

1(5) 

N 

K6) 

N 

KB) 

-2.00 

-1.80 

- 

_ 

_ 

_ 

_ 

- 

- 

_ 

_ 

_ 

-1.79 

-1.60 

- 

- 

- 

- 

- 

- 

2 

.50 

2 

.36 

-1.59 

-1.40 

2 

.85 

6 

.70 

3 

.46 

10 

.71 

3 

.66 

-1.39 

-1.20 

11 

1.18 

14 

1.13 

9 

.71 

14 

1.16 

16 

1.13 

-1.19 

-1.00 

20 

1.75 

29 

1.79 

9 

1.12 

12 

1.79 

13 

1.79 

-.99 

-.80 

18 

2.59 

5 

2.80 

9 

1.77 

22 

2.61 

23 

2.56 

-.79 

-.60 

33 

3.61 

31 

3.47 

20 

2.57 

14 

3.44 

21 

3.36 

-.59 

-.40 

18 

4.23 

13 

4.16 

18 

3.43 

21 

4.09 

26 

4.09 

-.39 

-.20 

30 

4.26 

45 

4.17 

25 

4.08 

32 

4.18 

24 

4.17 

-.19 

.00 

25 

3.85 

14 

3.90 

31 

4.17 

25 

3.88 

12 

3.78 

.01 

.20 

27 

3.41 

21 

3.40 

19 

3.87 

27 

3.32 

30 

3.36 

.21 

.40 

25 

3.19 

38 

3.14 

33 

3.42 

19 

3.14 

20 

3.14 

.41 

.60 

27 

3.30 

11 

3.23 

29 

3.14 

25 

3.21 

29 

3.20 

.61 

.80 

16 

3.39 

23 

3.35 

20 

3.22 

26 

3.34 

30 

3.34 

.81 

1.00 

18 

3.18 

20 

3.19 

17 

3.34 

16 

3.05 

16 

3.05 

1.01 

1.20 

13 

2.74 

16 

2.58 

19 

3.06 

16 

2.61 

12 

2.58 

1.21 

1.40 

6 

2.26 

5 

2.38 

6 

2.25 

12 

2.23 

10 

2.15 

1.41 

1.60 

9 

1.98 

5 

1.96 

10 

1.91 

3 

1.84 

10 

1.86 

1.61 

1.80 

- 

- 

- 

- 

3 

1.84 

4 

1.90 

1 

1.86 

1.81 

2.00 

2 

5.80 

4 

1.96 

1 

1.98 

- 

- 

2 

1.97 

Total 

Group 

300 

3.22 

300 

3.09 

300 

3.08 

300 

3.05 

300 

3.03 

Table  J 

Mean  Information  Values  (I)  at  Estimated  Achievement  Levels  (@) 
for  Subtest  6  under  All  Testing  Conditions 


6  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

HI 

N 

N 

1(0) 

N 

1(6) 

N 

1(5) 

N 

1(6) 

-2.00 

-1.80 

_ 

_ 

_ 

I 

.04 

_ 

_ 

_ 

_ 

-1.79 

-1.60 

1 

.25 

- 

- 

4 

.14 

2 

.15 

3 

.14 

-1.59 

-1.40 

5 

.43 

9 

.32 

4 

.42 

7 

.46 

4 

.31 

-1.39 

-1.20 

12 

1.23 

19 

1.03 

10 

.99 

11 

1.10 

9 

1.12 

-1.19 

-1.00 

20 

2.07 

7 

2.18 

13 

2.05 

13 

1.93 

12 

2.08 

-.99 

-.80 

10 

2.59 

24 

2.56 

14 

2.58 

9 

2.55 

13 

2.56 

-.79 

-.60 

18 

2.43 

9 

2.38 

29 

2.40 

23 

2.40 

21 

2.46 

-.59 

-.40 

25 

2.23 

26 

2.18 

19 

2.20 

26 

2.18 

16 

2.18 

-.39 

-.20 

32 

2.16 

18 

2.12 

34 

2.10 

32 

2.10 

48 

2.10 

-.19 

.00 

23 

2.13 

49 

2.10 

38 

2.07 

45 

2.08 

35 

2.07 

.01 

.20 

36 

2.14 

19 

2.13 

17 

2.12 

16 

1.12 

15 

2.12 

.21 

.40 

35 

2.29 

16 

2.28 

28 

2.25 

23 

2.26 

30 

2.26 

.41 

.60 

16 

2.49 

41 

2.48 

21 

2.42 

20 

2.43 

22 

2.41 

.61 

.80 

9 

2.66 

- 

- 

10 

2.60 

18 

2.60 

13 

2.60 

.81 

1.00 

14 

2.60 

18 

2.56 

21 

2.52 

17 

2.52 

26 

2.52 

1.01 

1.20 

26 

2.28 

- 

- 

9 

2.27 

15 

2.24 

11 

2.34 

1.21 

1.40 

12 

1.86 

45 

1.77 

17 

1.82 

15 

1.81 

17 

1.83 

1.41 

1.60 

5 

1.56 

- 

- 

8 

1.41 

5 

1.45 

4 

1.47 

1.61 

1.80 

- 

- 

- 

- 

3 

1.17 

3 

1.18 

l 

1.24 

1.81 

2.00 

1 

1.86 

- 

- 

- 

- 

- 

- 

- 

- 

Total 

Group 

300 

2.17 

300 

2.08 

300 

2.10 

300 

2.10 

300 

2.15 

26 


Table  K 

Mean  Information  Values  (I)  at  Estimated  Achievement  Levels  (@) 
for  Subteat  7  under  All  Testing  Conditions 


§  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

Hi 

N 

1(8) 

H 

1(0) 

N 

1(0) 

N 

1(6) 

N 

1(0) 

-2.00 

-1.80 

- 

• 

_ 

- 

_ 

_ 

_ 

1 

.06 

-1.79 

-1.60 

17 

.31 

- 

- 

5 

.18 

6 

.27 

7 

.23 

-1.59 

-1,40 

- 

16 

.69 

3 

1.28 

10 

1.13 

6 

1.11 

-1.39 

-1.20 

- 

- 

- 

- 

2 

3.07 

- 

- 

1 

3.26 

-1.19 

-1.00 

“ 

- 

- 

- 

- 

- 

1 

3.04 

- 

- 

-.99 

-.80 

6 

1.47 

- 

- 

3 

1.16 

10 

1.04 

6 

1.30 

-.79 

-.60 

22 

.73 

41 

.56 

23 

.58 

24 

.56 

20 

.57 

-.59 

-.40 

40 

.45 

14 

.45 

27 

.43 

34 

.42 

34 

.43 

-.39 

-.20 

31 

.65 

89 

.66 

41 

.58 

35 

.60 

42 

.61 

-.19 

.00 

48 

1.00 

- 

- 

38 

1.00 

26 

.96 

31 

.96 

.01 

.20 

21 

1.47 

25 

1.42 

39 

1.47 

37 

1.38 

35 

1.38 

.21 

.40 

23 

1.77 

30 

1.87 

23 

1.79 

18 

1.82 

18 

1.81 

.41 

.60 

43 

1.92 

82 

1.90 

31 

1.89 

30 

1.89 

39 

1.89 

.61 

.80 

38 

1.88 

- 

- 

34 

1.83 

38 

1.83 

30 

1.83 

.81 

1.00 

6 

1.84 

- 

- 

26 

1.79 

23 

1 .80 

20 

1.79 

1.01 

1.20 

- 

- 

- 

- 

2 

1.87 

4 

1.96 

1 

2.12 

1.21 

1.40 

- 

- 

- 

- 

- 

- 

- 

- 

1 

2.26 

1.41 

1.60 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

1.61 

1.80 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

1.81 

2.00 

5 

4.54 

3 

3.68 

3 

7.14 

4 

6.49 

8 

9.10 

Total 

Group 

300 

1.25 

300 

1.19 

300 

1.31 

300 

1.28 

300 

1.41 

Table  L 

Mean  Information  Values  (I)  at  Estimated  Achievement 
for  Subtest  8  under  All  Testing  Conditions 

Levels  (g) 

9  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

Hi 

N 

N 

1(0) 

N  1(0) 

N 

K0) 

N 

1(0) 

-2.00 

-1.80 

- 

- 

_ 

_ 

_ 

_ 

_ 

_ 

-1.79 

-1.60 

- 

- 

- 

- 

1  .20 

- 

- 

1 

.03 

-1.59 

-1.40 

6 

.09 

6 

.08 

6  .06 

5 

.09 

10 

.10 

-1.39 

-1.20 

29 

.73 

19 

.65 

7  .66 

17 

.59 

13 

.48 

-1.19 

-1.00 

14 

2.06 

5 

2.23 

16  2.29 

9 

2.60 

7 

2.78 

-.99 

-.80 

- 

- 

1 

3.82 

1  3.88 

5 

3.97 

3 

4.06 

-.79 

-.60 

22 

3.38 

- 

- 

14  3.33 

18 

3.24 

7 

3.44 

-.59 

-.40 

24 

2.30 

49 

2.18 

52  1.98 

37 

1.96 

41 

1.93 

-.39 

-.20 

36 

1.54 

49 

1.48 

30  1.46 

37 

1.46 

35 

1.46 

-.19 

.00 

41 

1.76 

20 

1.67 

33  1.66 

34 

1.68 

40 

1.66 

.01 

.20 

22 

2.35 

41 

2.18 

31  2.29 

28 

2.30 

24 

2.22 

.21 

.40 

24 

3.43 

25 

2.98 

29  3.24 

21 

3.27 

32 

3.16 

.41 

.60 

19 

4.52 

30 

4.36 

20  4.48 

23 

4.48 

20 

4.46 

.61 

.80 

12 

4.97 

12 

4.92 

16  4.91 

15 

4.92 

17 

4.92 

.81 

1.00 

8 

4.70 

4 

4.83 

10  4.75 

17 

4.77 

12 

4.75 

1.01 

1.20 

14 

4.65 

10 

4.74 

9  4.69 

14 

4.66 

11 

4.66 

1.21 

1.40 

6 

5.25 

2 

4.92 

6  5.10 

5 

5.06 

4 

5.14 

1.41 

1.60 

15 

5.31 

6 

5.33 

5  5.26 

3 

5.14 

10 

5.25 

1.61 

1.80 

2 

4.55 

10 

4.68 

4  4.82 

2 

4.84 

3 

4.57 

1.81 

2.00 

8 

2.96 

11 

2.11 

10  2.61 

10 

2.62 

10 

2.65 

Total 

Group 

300 

2.75 

300 

2.58 

300  2.73 

300 

2.76 

300 

2.70 

27 


Table  M 

Mean  Information  Values  (I)  at  Estimated  Achievement  Levels  ( §) 
for  Subtest  9  under  All  Testing  Conditions 


8  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

Hi 

N 

Ke> 

N 

I(§) 

N 

1(8) 

N 

1(6) 

N 

K§) 

-2.00 

-1.80 

- 

- 

- 

- 

- 

— 

_ 

_ 

-1.79 

-1.60 

- 

- 

- 

- 

3 

.29 

- 

- 

3 

.41 

-1.59 

-1.40 

3 

.85 

4 

.80 

5 

.94 

6 

.86 

6 

.92 

-1.39 

-1.20 

13 

1.85 

13 

1.56 

6 

1.71 

17 

1.71 

9 

1.38 

-1.19 

-1.00 

24 

2.89 

18 

2.83 

17 

2.94 

13 

2.77 

9 

2.83 

-.99 

-.80 

27 

3.94 

15 

3.87 

16 

3.84 

25 

3.88 

19 

3.93 

-.79 

-.60 

16 

4.55 

27 

4.49 

17 

4.50 

16 

4.49 

17 

4.48 

-.59 

-.40 

21 

4.79 

28 

4.70 

30 

4.60 

21 

4.68 

29 

4.69 

-.39 

-.20 

28 

4.75 

28 

4.66 

33 

4.66 

28 

4.67 

31 

4.66 

-.19 

.00 

29 

4.78 

26 

4.69 

29 

4.68 

37 

4.69 

24 

4.68 

.01 

.20 

27 

4.95 

25 

4.86 

33 

4.94 

31 

4.92 

31 

4.90 

.21 

.40 

22 

5.23 

23 

5.23 

23 

5.25 

20 

5.23 

30 

5.23 

.41 

.60 

13 

5.24 

24 

5.24 

22 

5.19 

27 

5.19 

26 

5.22 

.61 

.80 

23 

4.89 

10 

4.76 

20 

4.74 

14 

4.70 

20 

4.68 

.81 

1.00 

16 

4.16 

20 

4.13 

17 

4.18 

12 

4.14 

16 

4.18 

1.01 

1.20 

12 

3.66 

13 

3.45 

10 

3.54 

14 

3.57 

11 

3.49 

1.21 

1.40 

12 

3.39 

9 

3.33 

6 

3.28 

7 

3.31 

10 

3.30 

1.41 

1.60 

4 

3.32 

10 

3.22 

8 

3.20 

8 

3.22 

7 

3.22 

1.61 

1.80 

4 

3.34 

4 

3.21 

2 

3.11 

1 

3.18 

- 

- 

1.81 

2.00 

6 

3.76 

3 

4.85 

3 

5.01 

3 

5.52 

2 

4.84 

Total 

Group 

300 

4.26 

300 

4.23 

300 

4.33 

300 

4.24 

300 

4.32 

Table  N 

Mean  Information  Values  (I)  at  Estimated  Achievement  Levels  (6) 
for  Subtest  10  under  All  Testing  Conditions 

0  Range 

Conven¬ 

tional 

Test 

Intra- 
Sub  test 
Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

Hi 

N 

1(0) 

N 

K8) 

N 

1(0) 

N 

1(8) 

N 

He) 

-2.00 

-1.80 

- 

_ 

- 

• 

- 

_ 

_ 

- 

_ 

-1.79 

-1.60 

- 

- 

- 

- 

1 

.14 

2 

.13 

- 

- 

-1.59 

-1.40 

7 

.30 

6 

.47 

1 

.60 

2 

.30 

4 

.47 

-1.39 

-1.20 

1 

1.02 

- 

- 

4 

.82 

- 

- 

9 

.75 

-1.19 

-1.00 

14 

1.11 

18 

1.07 

10 

1.06 

8 

1.06 

12 

1.06 

-.99 

-.80 

27 

1.09 

17 

1.03 

16 

1.03 

35 

1.04 

13 

1.03 

-.79 

-.60 

20 

1.20 

17 

1.11 

26 

1.19 

28 

1.23 

29 

1.18 

-.59 

-.40 

22 

1.57 

29 

1.44 

34 

1.57 

21 

1.50 

29 

1.57 

-.39 

-.20 

30 

2.18 

31 

2.09 

29 

2.06 

34 

2.10 

28 

2.10 

-.19 

.00 

35 

2.74 

72 

2.64 

34 

2.65 

38 

2.57 

35 

2.65 

.01 

.20 

31 

3.08 

3 

2.97 

36 

2.94 

23 

2.94 

23 

2.95 

.21 

.40 

31 

2.99 

32 

2.89 

25 

2.88 

27 

2.88 

24 

2.87 

.41 

.60 

23 

2.77 

28 

2.59 

27 

2.67 

33 

2.68 

31 

2.63 

.61 

.80 

21 

2.66 

il 

2.57 

16 

2.54 

12 

2.54 

19 

2.54 

.81 

1.00 

17 

2.89 

12 

2.78 

8 

2.80 

13 

2.83 

13 

2.79 

1.01 

1.20 

7 

3.47 

8 

3.64 

12 

3.44 

8 

3.57 

11 

3.55 

1.21 

1.40 

3 

4.43 

5 

4.54 

8 

4.37 

8 

4.42 

9 

4.40 

1.41 

1.60 

7 

5.59 

3 

5.16 

4 

5.54 

3 

5.50 

4 

5.31 

1.61 

1.80 

- 

- 

5 

6.47 

3 

6.77 

2 

6.58 

4 

6.27 

1.81 

2.00 

4 

9.23 

3 

9.58 

6 

10.52 

3 

8.17 

3 

10.74 

Total  Group 

300 

2.46 

300 

2.40 

300 

2.53 

300 

2.33 

300 

2.42 

-  28 


Table  0 

Mean  Inforaation  Values  (I)  at  Estimated  Achievement  Levels  (6) 
for  Subtest  11  under  All  Testing  Conditions 


6  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

Lo 

Hi 

N 

1(0) 

N 

ITC) 

N 

K0) 

N 

I W> 

N 

1(6) 

-2.00 

-1.80 

- 

- 

- 

- 

• 

_ 

• 

3 

.20 

-1.79 

-1.60 

4 

.60 

2 

.54 

5 

.56 

- 

- 

5 

.84 

-1.59 

-1.40 

8 

1.12 

8 

.93 

l 

.91 

16 

.94 

5 

.84 

-1.39 

-1.20 

13 

1.96 

19 

1.70 

1 

1.76 

12 

1.72 

14 

1.85 

-1.19 

-1.00 

21 

3.33 

14 

3.36 

1 

3.14 

21 

3.41 

12 

3.06 

-.99 

-.80 

22 

5.12 

24 

5.21 

2 

5.09 

22 

5.19 

23 

5.16 

-.79 

-.60 

25 

6.68 

24 

6.81 

1 

6.78 

26 

6.85 

22 

6.90 

-.59 

-.40 

21 

7.26 

29 

7.18 

2 

7.18 

31 

7.18 

35 

7.19 

-.39 

-.20 

17 

7.06 

20 

7.03 

2 

7.04 

20 

7.04 

24 

7.04 

-.19 

.00 

32 

7.20 

24 

7.20 

2 

7.17 

24 

7.22 

26 

7.19 

.01 

.20 

28 

7.93 

30 

8.08 

2 

7.91 

21 

8.01 

19 

7.91 

.21 

.40 

22 

8.58 

21 

8.54 

1 

8.53 

11 

8.53 

18 

8.58 

.41 

.60 

14 

7.86 

21 

7.45 

1 

7.35 

18 

7.61 

23 

7.56 

.61 

.80 

11 

5.36 

10 

5.49 

1 

5.46 

17 

5.41 

17 

5.40 

.61 

1.00 

23 

3.70 

11 

3.27 

2 

3.40 

16 

3.22 

16 

3.60 

1.01 

1.20 

20 

2.73 

15 

2.49 

1 

2.58 

15 

2.53 

17 

2.55 

1.21 

1.40 

10 

2.15 

5 

2.13 

1 

2.10 

6 

2.09 

9 

2.08 

1.42 

1.60 

5 

2.09 

16 

1.99 

1 

2.00 

12 

1.99 

7 

2.00 

1.61 

1.80 

2 

2.52 

2 

2.33 

2.32 

5 

2.32 

3 

2.46 

1.81 

2.00 

2 

2.92 

5 

2.72 

2.77 

7 

2.72 

2 

2.95 

Total  Group 

300 

5.50 

300 

5.55 

30 

5.29 

300 

5.28 

300 

5.55 

Table  P 

Mean  Inforaation  Values  (I)  at  Estiasted  Achievement 
for  Subtest  12  under  All  Testing  Conditions 

Levels 

(§) 

fi  Range 

Conven¬ 

tional 

Test 

Intra- 

Subtest 

Item 

Selection 

Intra-Subtest  Item  Selection 
with  Inter-Subtest  Branching 
Multiple  No.  of 

R  Items  Random 

to 

Hi 

N 

K0) 

N 

1(0) 

N  1(6) 

N 

1(0) 

N 

1(8) 

-2.00 

-1.80 

- 

- 

- 

- 

_  — 

_ 

_ 

_ 

_ 

-1.79 

-1.60 

- 

- 

- 

- 

- 

- 

- 

- 

- 

-1.59 

-1.40 

1 

.04 

- 

- 

- 

1 

.04 

1 

.03 

-1.39 

-1.20 

7 

.11 

2 

.11 

2  .13 

3 

.15 

3 

.12 

-1.19 

-1.00 

13 

.29 

11 

.28 

11  .27 

12 

.29 

9 

.26 

-.99 

-.80 

31 

.53 

26 

.52 

20  .46 

23 

.51 

18 

.54 

-.79 

-.60 

33 

.86 

26 

.96 

26  .84 

26 

.89 

32 

.86 

-.59 

-.40 

26 

1.25 

8 

1.39 

41  1.25 

39 

1.26 

37 

1.25 

-.39 

-.20 

21 

1.77 

82 

1.58 

30  1.69 

34 

1.66 

30 

1.69 

-.19 

.00 

34 

2.09 

11 

1.99 

32  2.04 

30 

2.03 

25 

2.03 

.01 

.20 

36 

2.31 

46 

2.28 

35  2.29 

31 

2.32 

29 

2.32 

.21 

.40 

21 

2.60 

13 

2.71 

19  2.55 

22 

2.62 

19 

2.60 

.41 

.60 

23 

2.84 

23 

2.79 

31  2.79 

19 

2.81 

28 

2.79 

.61 

.80 

8 

2.82 

15 

2.82 

17  2.77 

6 

2.76 

14 

2.78 

.81 

1.00 

12 

2.63 

- 

- 

20  2.51 

24 

2.55 

27 

2.53 

1.01 

1.20 

17 

2.32 

36 

2.37 

7  2.20 

17 

2.29 

14 

2.30 

1.21 

1.40 

11 

2.13 

- 

- 

8  2.04 

8 

2.04 

10 

2.02 

1.41 

1.60 

4 

2.16 

- 

- 

- 

1 

2.13 

1 

2.10 

1.61 

1.80 

2 

2.88 

- 

- 

- 

1 

2.70 

1 

2.92 

1.81 

2.00 

- 

- 

1 

6.52 

1  7.86 

3 

7.69 

1 

6.30 

Total 

Group 

300 

1.75 

300 

1.81 

300  1.84 

300 

1.83 

300 

1.85 
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